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by 

JAMES AFARIN 

The managers in Federal agencies are challenged to control the extensive 
activities in government and still provide high-quality products and services to 
the American taxpayers. Considering today’s complex social and economic 
environment and the $3.8 billion daily cost of operating the Federal 
Government, it is evident that there is a need to develop decision-making tools 
for accurate resource allocation and total quality management. 

The goal of this thesis is to provide a methodical process that will aid 
managers in Federal Government to make budgetary decisions based on the 
cost of services, the agency’s objectives, and the customers’ perception of the 
agency’s product. 

A general resource allocation procedure was developed in this study that 
can be applied to any government organization. A government organization, 
hereafter the "organization," is assumed to be a multidivision enterprise. This 
procedure was applied to a small organization for the proof of the concept. 
This organization is the Technical Services Directorate (TSD) at the NASA 
Lewis Research Center in Cleveland, Ohio. 



As a part of the procedure, a nonlinear programming model was developed 
to account for the resources of the organization, the outputs produced by the 
organization, the decision-makers views, and the customers’ satisfaction with 
the organization. 

The information on the resources of the organization was acquired from 
current budget levels of the organization and the human resources assigned 
to the divisions. The outputs of the organization were defined and measured 
by identifying metrics that assess the outputs, the most challenging task in this 
study. 

The decision-maker’s views are represented in the model as weights 
assigned to the various outputs and were quantified by using the analytic hier- 
archy process. The customers’ opinions regarding the outputs of the organiza- 
tion were collected through questionnaires that were designed for each division 
individually. Following the philosophy of total quality management, information 
on customers’ satisfaction is presented in the model as the quality of output. 

The model is a nonlinear one whose objective is to maximize customers’ 
satisfaction such that the total cost of operation does not exceed the 
organization’s budget. This model represents a structured approach or policy 
mechanism, at the agency level, to make capital investment decisions based 
on the priorities of the agency and the quality of outputs. This procedure 
applied to TSD resulted in a resources allocation scheme that was reasonable 
and acceptable to the decision-makers and, as expected, dependent on the 
assumptions and accuracy of the data used in the model. 
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PART ONE 


INTRODUCTION AND MODEL FORMULATION 

This part contains two chapters. In Chapter 1, before developing a 
mathematical model for optimum resource allocation, the need for this effort is 
justified. Next, the history of the budgeting system is considered and the 
contributions of the thesis are outlined. In Chapter 2 the mathematical model 
is introduced and a general problem formulation is developed. Next, a more 
specific model is introduced and all the assumptions are justified. 
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Chapter 1 

Introduction and Background 

In this chapter, the need for this study is justified. In Section 1.1, a 
background for the problem is offered and a foundation for the following 
chapters is presented. In Section 1.2, the historical background of the 
budgeting system is discussed, and in Section 1.3, the contributions of this 
work are highlighted. 

1.1 Introduction 

The national debt is over $4 trillion. It took more than 2 centuries, 
1776-1981, to reach $1 trillion of debt and only 12 years to add $3 trillion. If 
this trend continues to the year 2000, the national debt will reach $13 trillion 
and 95% of all personal income taxes will be used to pay only the interest on 
the debt [Grace 1988]. 

A major contributor to the size of the deficit is government waste. Martin 
Gross identified 75 different areas where significant amounts of money are 
wasted [Gross 1992]. He argues that less spending, not higher taxes, is the 
best way to reduce the deficit [Grace 1989]. The Wall Street Journal in the 
November 24, 1 992, issue reported that budget experts are urging President 
Clinton to reduce government waste to solve the nation’s deficit problem 
[Wartzman 1992]. 
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Presently, the decision-makers are not provided with the opportunity to 
examine the objectives of various programs. Interagency trade-offs are difficult 
to examine, and the link between cost and services is hard to discern. Cuts 
are imposed without recognizing their impact on various services. Agencies 
are frequently expected to absorb cuts and still maintain the current level of 
services. 

A resource allocation procedure that examines the government’s goals and 
measures the output of the organization will help managers to use resources 
in the most efficient manner possible, eliminate waste, and consequently 
reduce the deficit. Traditionally, the resources are allocated on the basis of 
incremental change to the categories of expenses known as line items. Each 
year at the beginning of the budget process, the previous year’s budget is 
selected as the model for the present year’s budget. Experts contend that this 
is a mistake because the previous year’s budget is the result of thousands of 
errors that accumulated over prior years [Gross 1992]. During the course of 
history, several attempts have been made to reform the budget process in a 
manner that reflects government objectives and measures the production of 
government. Budgeting systems, such as performance budgeting (PB) in the 
1 940s, the planning, programming, and budgeting system (PPBS) in the 1 960s, 
and zero-base budgeting (ZBB) in the 1970s, were introduced to change the 
incremental budget, but they were not successful (see Section 1 .2). 
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The objective of this study was to develop a decision-making tool for 
optimizing the allocation of resources for the purpose of maximizing the 
customers’ satisfaction. This tool is designed for all government organizations 
at agency and subagency levels regardless of their size. The objectives for 
developing a decision-making tool for managers in the Federal Government are 
(1) to assist decision-makers in making consistent decisions; (2) to provide 
decision-makers with information on the trade-offs and pitfalls of their decisions; 
(3) to provide decision-makers with a road map and systematic approach to 
arriving at a decision; and (4) to document the process of decision-making for 
future reference and to maintain consistency [Hobbs et al. 1992]. 

This thesis is unique because it borrows the notion of measuring 
productivity from PB, focusing on objectives from PPBS, and considering 
different alternatives from ZBB. Then, it employs optimization methods and 
customer perception in deciding the best resource allocation scheme. It also 
institutes a standard format for making decisions that follows the classical 
model of rational decision-making, requiring that managers identify objectives 
and alternatives, select criteria, and choose alternatives based on the criteria. 

An actual application of the decision-making tool requires specific 
information about the organization under study. In this study, the mathematical 
model was developed for a typical organization, but assumptions and the model 
input, such as type of resources, parameters, and decision variables and their 
values, were specifically tailored for the Technical Services Directorate (TSD). 
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TSD is one of nine organizations called directorates that make up the NASA 
Lewis Research Center. These directorates have specific functions, and their 
efforts are concentrated and focused to satisfy the mission of Lewis, which is 
presented in Section 3.1. The directorate under consideration, TSD, provides 
engineering, technical, and environmental services to support both experimental 
research and facilities. A major portion of this study is dedicated to developing 
procedures for gathering data that will help decision-makers to make right 
assumptions in the various stages of model development. 

The organization is not profit oriented and is not therefore controlled by the 
free market. At the same time, its very existence is dependent on the optimum 
use of its resources and the quality of the goods and services it provides to the 
taxpayers. The organization was created to fulfill a specific need of society and 
is bound by the regulations imposed by legislative bodies, such as Congress 
and state and local governments. 

Optimum resource allocation means identifying and evaluating all the 
activities in an organization and then increasing resources only among those 
that, within the given fiscal constraints, laws, and executive orders, will 
maximize the effectiveness of the organization. Effectiveness is defined as the 
degree of success achieved in attaining predetermined goals or responding to 
the demand placed on the organization [Kinlaw 1987]. 

The textbook approach to this problem is to associate each activity with a 
measurable output. The tangible or intangible benefits accrued to the 
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organization from a unit of output are balanced against the cost of producing 
the output. An overall measure of goodness is defined as the objective 
function, and the resource allocation is then continued so as to maximize the 
value of the objective function. For two or more objectives, a multiobjective 
ranking and optimization method must be used to make allocation decisions. 

This technique is adequate if we are dealing with a manufacturing company 
that produces tangible output, such as automobiles, light bulbs, or doors. 
When we are faced with an organization in a large Federal research-and- 
development (R&D) establishment, such as NASA, the issues become fuzzy, 
and difficulties arise in measuring the output and determining the objective 
functions of the organization. For example, the output may be providing 
planning for satisfying the R&D facility’s requirements or ensuring compliance 
with environmental laws and regulations. The objective function could be 
maximizing customer satisfaction or organization output. 

The problem is fundamentally the same as analyzing a manufacturing firm 
that counts widgets to determine its level of output. But since we cannot 
optimize what we cannot measure, a critical element of optimization in this 
research is the ability to identify and measure the output of the organization. 
The complexity of the problem now becomes evident. 

A system that only measures the output of an organization does not provide 
the total picture. The quality of the output is just as important as the level of 
the output. Therefore, we are also faced with accounting for quality in 


7 


measuring the effectiveness of organization output. In the case of NASA, 
where the level of resources is fixed, the questions are as follows: What is the 
definition of quality? What is the cost of maintaining the level of quality and the 
quantity of the output? Should the quantity be reduced to improve quality? 
What level of quality is acceptable in our production? 

For answers to some of these questions, we turn to total quality 
management (TQM). TQM is a comprehensive customer-focused system with 
seven major elements based on criteria of the President’s Award for Quality 
[Lewis 1991]. These elements are top management leadership and support, 
strategic planning, focus on the customer, measurement and analysis, 
commitment to training and recognition, employee empowerment and 
teamwork, and quality assurance. 

In the TQM context, the definition of quality is meeting customer 
requirements in developing a product or providing a service the first time and 
every time [Department of Commerce, 1989]. It is recognized that this 
definition is subjective and may vary from one customer to another. A great 
deal of communication is required to produce a specific definition of quality for 
an organization. To develop a model that will successfully display the level of 
quality obtained at any given time, a unit for measuring the quality should be 
created for every division of the organization. This measuring scale will enable 
the organization’s managers to gauge their decisions on the basis of improving 
quality. 
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Although optimization of resource allocation is a classic problem and is 
analyzed in many publications [Lasdon 1970, Saaty 1982, Nagel and Long 
1 989], the development of a model is seldom investigated specifically when the 
subject is a Federal Government organization with a specific requirement for 
implementing TQM. The tool developed in this study consists of a 
mathematical model of the organization that includes all the activities, fiscal 
constraints, and limitations in a TQM framework. The concept of TQM will be 
reflected in all aspects of this research where appropriate. In other words, this 
model will be an amalgamation of mathematical optimization, resource 
allocation techniques, and the management system of TQM. 

The cornerstone of this work will be the development of measuring sticks 
for quantifying the output and quality of every activity in the organization. 
These data, in mathematical form, will become the objective functions and 
decision variables for the optimization problem. The restriction on budget, 
manpower, and physical aspects of the organization will become part of the 
constraints of the problem. Constraints also define production functions, which 
convert resources into output. The final result is expected to be a flexible 
decision-making tool that will aid the director in allocating the resources of the 
organization in the best possible manner to maximize the quality and the 
quantity of the output. 
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1.2 Public Budgeting Systems 

A study on fiscal federalism reveals that government collects more than 47 
percent of all personal income. The Federal Government collects 27 percent, 
the states 8 percent, and local governments 1 2 percent [Advisory Commission 
1990]. Federal Government spending is divided into controllable and 
uncontrollable programs. The entitlement programs are considered as 
uncontrollable. Under these programs, those who meet the statutory standards 
are legally entitled to receive certain benefits regardless of the budget 
circumstances. Unless the entitlement laws are changed, the Federal budget 
makers must look at the controllable portion of the budget to implement fiscal 
policies. Therefore, organizations such as the Department of Defense, the 
Department of State, and NASA are under tighter budget constraints in 
performing their missions. 

Excluding the entitlement programs, budgetary decisions are made at 
macro- and microlevels. The macrolevel decisions are made in the Office of 
Management and Budget and in the Congress, where legislators are concerned 
with the size of the Government’s budget, spending levels, and how much 
taxing should be imposed [LeLoup 1988]. For example, Congress decides the 
portion of the budget that is appropriated to the Department of Defense. Also, 
it decides how much taxes should be increased or how big a deficit it is willing 
to accept to support the appropriation. These decisions are based on political 
issues, the mood of the nation, and international circumstances. 
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With the exception of the Department of Defense, which partially employs 
the planning, programming, and budgeting system (explained later), the agency 
and subagency budgeting process is in practice incremental, not 
comprehensive [Posner 1993]. An agency’s budget is almost never reviewed 
as a whole every year. Instead, it is based on last year’s budget with attention 
focused on a range of increases or decreases. This concept of incrementalism 
was first published by Simon [1958] in his classic study, Administrative 
Behavior , and then by several other researchers (Lindblom [1961,1966] and 
Davis et al. [1966]). 

Resource allocation is one of the most important decision-making 
processes in government organizations. Although Congress is charged with 
making these decisions at a macrolevel, agencies and organizations are 
responsible for using the appropriated funds in the most effective manner 
possible. Public budgeting consists of the techniques that decision-makers at 
all levels of government can use to justify and allocate their resources. 
Through the course of history, many budgeting systems were invented and 
implemented. In the following paragraphs, a brief explanation of the various 
budgeting systems is presented. 

There was no formal budgetary procedure until the Budget and Accounting 
Act of 1921 [Howard 1973]. Before this time, Federal agencies would submit 
budget requests to the Congress directly without Presidential influence. After 
World War I, President Wilson initiated the idea of executive control on the 
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budget mechanism but met strong resistance from Congress, which feared that 
this would give the President too much authority. Finally, by 1921 , the Budget 
and Accounting Act was enacted, establishing the Bureau of the Budget 
(renamed the Office of Management and Budget in 1970), an arm of the 
executive branch that controls the budgeting process. The Budget and 
Accounting Act also established the General Accounting Office, which reports 
directly to Congress [Hyde 1992]. 

The 1920s was an era of fiscal control and responsibility. By the 1930s, 
the budget system was refined, and the line-item budget system was born 
[Babunakis 1976]. This refinement was the result of a concentrated effort by 
the budget bureaus on the spending process rather than on planning or 
developing programs. 

In 1940, V.O. Key, Jr., wrote the famous article, ‘The Lack of Budgetary 

Theory." It asked the question, "On what basis shall it be decided to allocate 

X dollars to activity A instead of activity B?" Key wrote, 

The budget-maker never has enough revenue to meet the request of all 
spending agencies and he must decide how scarce means shall be allocated 
to alternative uses [Key 1940]. 

During the 1940s and 1950s, a change from fiscal control budgeting to 
management control budgeting occurred. In 1949, by recommendation of the 
Hoover Commission, the National Security Act Amendment law was signed, 
and performance budgeting emerged as a new budgeting system. The focus 
of this scheme was on the activities of the government rather than on the 
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objectives or expenditures. This change in emphasis was a budgetary turning 
point where management gained control from the accounting officials in 
developing the budget [Grossbard 1971]. 

The performance budget failed in the 1950s because it was purely 
mechanical and replaced clarity with complexity [Schick 1971]. The main 
concern of the performance budget was with efficiency; the justification for the 
activity was ignored. Its goal was to maintain efficiency rather than to serve 
the needs of taxpayers [Knezevich 1973]. Because of this, it did not receive 
the necessary support from the lawmakers and administrators. 

In the 1 960s, government activities increased, and the emphasis was on 
identifying major objectives and programs to support these objectives [Novick 
1968]. In 1960, the RAND Corporation published The Economics of Defense 
in the Nuclear Age [Hitch and McKean I960]. The Kennedy administration 
reviewed this document, and as a result, the planning, programming, and 
budgeting system (PPBS) was adopted by the Department of Defense in 1961 
under Robert McNamara [Knezevich 1973]. 

PPBS had two major objectives: (1) developing the objectives of the 
organization and (2) devising a system for achieving those objectives [Schultze 
1968]. PPBS was concerned with clarifying the objectives of government 
programs and monitoring the relationship between the output of the program 
and its objective. Another objective of PPBS was to establish the total cost of 
the program in the future. This was particularly important, since PPBS 
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searched for the most effective alternative of obtaining the program’s 
objectives. 

Although President Johnson claimed that PPBS was "a very new and 
revolutionary system" [Bureau of the Budget, 1968], the history of the PPBS in 
the Federal Government dates back to wartime control systems introduced by 
the War Production Board in 1942. In the private sector, it can be traced to 
1920, when it was introduced in General Motors and perhaps even earlier in 
Dupont [Novick 1 970]. A former assistant of the Bureau of the Budget testified 
that "every element of procedure and organization in PPBS can be found here 
and there within the Executive Branch prior to August 1965" [Comptroller 
General 1969]. This is the closest budgeting system to this research. The 
difference is that PPBS used cost-benefit analysis based on assumptions of 
future cost rather than using decision-maker’s value to rank alternatives. PPBS 
did not input the decision-maker’s judgment and did not account for the quality 
of output. Also, PPBS transferred the decision-making function to the 
production personnel and away from the managers. 

PPBS may have been the optimal method for the Department of Defense, 
but it was not workable for the entire Federal Government because the 
Congress refused to use it. The executive branch had required agencies to 
prepare two separate budgets: the line-item budget, and the PPBS budget. 
Congress only considered the line-item budget. The agencies were then forced 
to cross-reference the approved line items back to the PPBS budget [Rabin 
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1 988]. Owing to the inherent complications of dual budget systems, the PPBS 
was finally abandoned by the Nixon administration. 

Budget reforms in the 1970s resulted from competition between the 
legislative and executive branches over controlling budgets and spending. 
Also, at this time, resources were diminishing because of the lack of both 
economic growth and support for tax increases [Hyde 1992]. 

By 1977, in the Carter administration, zero-based budgeting (ZBB) was 
launched. In this system, every expenditure must be justified from the 
beginning of every year like a new expenditure. Every agency was required to 
rationalize each government program, new or existing, in its entirety each time 
an annual budget was formulated [Executive Office 1977]. Every budget 
proposal justified the expenditure for all new and existing projects on the basis 
of reevaluated goals, methods, objectives, and resources [Merewitz and 
Sosnick 1971]. 

On April 19, 1977, the Office of Management and Budget (OMB) issued 
budget preparation instructions for ZBB. Under these rules, managers of 
Federal departments had to identify and rank decision packages at four funding 
levels for each program: (1) The minimum level, below which the program 
would not be viable; (2) the maintenance level, that which is required to 
continue the existing level of operation without changing policy; (3) the 
intermediate level, which is a point between the minimum and maintenance 
levels; and (4) the improvement level, which expands the current level of 
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operation by requiring additional funding. In total, four budgets had to be 
prepared for every item every year [Axelrod 1988]. The ZBB implementation 
procedures significantly increased the workload of the agencies and 
overwhelmed government managers and were terminated by the Reagan 
administration. 

The 1980s was a decade when process "Mickey Mouse." The media 
labeled submitted budgets "dead on arrival" because they were based on 
unrealistic economic assumptions [Hyde 1992]. The 1985 Budget and 
Emergency Deficit Control Act (also known as the Gramm-Rudman-Hollings 
Act) was created with the singular goal of reducing the deficit and was passed 
within 90 days without a hearing [Collender 1990]. 

1.3 Contributions and Organization of Thesis 

The current budgeting system in the Federal Government considers 
resource allocation as a set of single-objective problems. The present budget 
system at agency level is in practice incremental, not comprehensive. No 
structured process exists that compares and ranks alternatives according to a 
set of criteria in a budgeting system. 

The main contribution of this study is the development of a systematic 
process to appropriate resources to various alternatives on the basis of 
performance, quality, and decision-makers input. The state of the art in the 
budgeting process of a government organization is advanced by incorporating 
the customers’ input and the organization’s objectives and effectiveness into 
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the allocation of resources. Among other contributions, the quantification of the 
organization’s output and its customer satisfaction is the most important one. 
Although developing metrics and measuring output and customer satisfaction 
are difficult, it was the start of a new philosophy in the government organization 
which articulates that customers’ opinions and the performance of the 
organization matter. 

Other contributions are the use of mathematical programming techniques 
in order to consider all alternatives in a comprehensive and consistent manner 
and the use of optimization techniques to aid managers in making the 
allocation decisions. In this thesis, the performance of the organization, the 
views of the customers, the preference of the decision-makers, and the budget 
limitation are linked through a nonlinear mathematical model that allows 
managers to examine various resource allocation schemes before disbursing 
funds. 

The remaining chapters of the thesis explain the different stages of the 
resource allocation process. In Chapter 2, the mathematical model of the 
organization is developed and all the parameters and variables are explained. 
Chapter 3 introduces the NASA mission and the TSD objective, which are used 
in the model. Chapter 4 defines the quality of the output and the method of 
assessing the quality. In Chapter 5, the boundaries of the outputs are acquired 
and the weights of the outputs are estimated. 
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In Chapter 6, the resource use functions are estimated, and in Chapter 7, 
the results of the TSD case study are presented. Conclusions are given in 
Chapter 8, followed by appendices and references. Appendix A introduces the 
quality dimensions that were used in developing the questionnaires for TSD. 
Appendix B describes the analytical hierarchy process, and Appendix C 
presents the results of implementing the model using TSD data. Appendix D 
displays the TSD functions and required resources, and Appendix E presents 
the questionnaires. 



Chapter 2 

Mathematical Modeling 


In this chapter, the concept of mathematical modeling and its use in this 
thesis are defined. The parameters and variables are defined and methods for 
estimating parameters are discussed. This information is necessary for using 
any optimization technique. In Section 2.1, mathematical modeling is defined 
and the phases of creating the model are described. In Section 2.2, a general 
formulation for the mathematical model is introduced, and in Section 2.3, a 
specific formulation for the government organization is presented. 

2.1 Introduction 

Mathematical modeling is the procedure of representing the behavior of a 
real system by sets of mathematical relationships. Modeling is the first step in 
the optimization process and provides a powerful tool for synthesizing, 
analyzing, planning, and controlling a complex system. In optimization 
techniques, mathematical models are formulated to determine values of 
decision variables that produce an optimum measure of goodness [Gill et al. 
1989]. 

Formulating the problem depends on the availability of reliable information 
and the ability to structure its relevant aspects. According to Singh and Titli 
[1978], the creation of a mathematical model has two phases: structure 
determination and parameter estimation. The first phase is to select a structure 
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for the model on the basis of the physical features of the system and the 
desired level of accuracy. The second phase is to assign values to the 
decision variables in the model structure such that the chosen structure 
delineates the system under consideration. 

2.2 General Problem Formulation 

In this section, the problem is transformed into a mathematical format in 
order to be analyzed and the model is developed for a multidivision 
organization. This is an optimization model where the objective is to maximize 
customer satisfaction. A measure of customer satisfaction, as a function of the 
quality and quantity of the organization’s output, is developed and maximized 
in the model. 

The optimization program was developed for the entire organization and not 
for individual divisions. The output was measured by a precise definition of a 
measure that counts the product. Quality was determined by the customers’ 
general opinions on the quality of divisions’ outputs, which were acquired 
through a questionnaire. There are also constraints on resource use and levels 
of production. 

Let X ijlt be the amount of output j produced by division i at quality level I 
using technology t. Technology, here, means a process that combines a 
particular mix of inputs to produce an output of a given quality. In a general 
case, some of the output produced by division i may be an input for division m, 
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and at the same time division i may receive input from division n. Without loss 
of generality, the formulation can be written for division i with vector X, = {X ijlt }. 

Let R,(Xj) = {R jjltk Vk} be the vector of resource type "k" that is used to 
produce X,, and let XniW = {XnrrrrM} be the vector of output j' from another 
division m at quality level I' using the technology f that is used as input to 
division i, where i * m. Let 7 ^ - {Z^} be the output vector of division i that 

customers receive directly. Total output X is divided into X m Xm and Z,. 

These concepts are shown in Figure 2.1 . 


Y. 

i m 


Y 


m i 


R 



Figure 2.1 Division i Input and Output. 


A general formulation is 


Max F(Zj Vi) 


such that, 


Rjp^Xjj,,) < B k Vk 
X| = Z, + X m Y,.(XJ Vi 


( 2 - 1 ) 


(2-2) 

(2-3) 
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All variables are nonnegative and bounded, where F( ) is the overall customer 
satisfaction as a function of quality and quantity. In the above formulation, it 
is assumed that the total resources that constitute the annual budget are fixed. 
Also, the allocation of resources within a division is the division chiefs decision. 
Furthermore, resources are assumed to be segregated into k types. This 
assumption is reasonable because, per government regulations, some of the 
resources are not interchangeable. For instance, the institutional budget 
(utilities, maintenance, etc.) cannot interchange with the R&D budget. 

A common implementation of equations (2-1) to (2-3) is to assume 
proportionality in all the functions. This assumption transforms them to 


F(Zj Vi) = Sb^ 

(2-4) 

Rpk(Xijlt) = a .ittk^ijlt 

(2-5) 

Y mi jit(X m ) = ZjTf^mijjll'tt'^mi'IT 

(2-6) 


where bj, otp*. and are the appropriate constants. This fixed-proportions 
approach is used frequently in economic input-output studies [Leontief 1966]. 
Leontief in his book Input-Output Economics used the same type of input- 
output analysis as equations (2-5) to (2-6) and argued that each economic 
sector requires inputs from other sectors that are assumed to be proportional 
to the first sector’s output. He also cited, in Chapter 7 of his book, over 35 
papers and articles that use the same proportionality concept to model 
allocation of resources for production of an economy’s output. This work 
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establishes a precedent for using the proportionality assumption for the input- 
output relations of a division. 

2.3 Specific Problem Formulation 

In this section, a specific model for this thesis based on the general model 
in the previous section is developed. Several assumptions were made to 
design a specific model that is practical and reliable for a government 
organization. I recognize that many of the assumptions are simplistic; but the 
purpose of the thesis is to demonstrate what can be learned from such a 
model. The thesis can then be a basis for developing a model that could be 
used for actual decisions. Such effort would require months of investigation, 
decision-makers commitment, and the scrutiny of affected parties inside and 
outside the agency. Below, I will describe and justify each assumption. 

2.3.1 Resource Mix Assumption, Index t 

The index t represents a technology that uses a unique mix of inputs for 
producing X jjlt in the model. For example, it is possible that some of output j 
from division i at quality I was produced by using in-house resources (t = 1) 
and that some of it was purchased from a local vendor (t = 2) during the period 
when the model was executed (3 to 6 months). Alternatively, different values 
of t can represent different ratios of civil servants and student interns. For the 
sake of simplicity and demonstration, I assumed that each output j can only be 
produced by one mix of resources — i.e., resources cannot be substituted for 
each other. Therefore, I eliminated the index t from all variables X, Y, and Z. 
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2.3.2 Objective Function Assumption F(Z,„) 

The overall customer satisfaction F represents the customers’ evaluation 
of the quality and quantity of an output. To illustrate my development of F(Zjj|), 
assume that division i has five outputs that can be produced in three quantity 
levels (low, medium, and high). Assume five quality dimensions (see Section 
4.3), such as communication, timeliness, and reliability, that can be measured 
on a scale of 1 to 5. 

Now, I would have to develop a utility function for each output in division i 
that represents the customers’ evaluation of different quantity levels (low, 
medium, and high) and every quality dimension (e.g., communication and 
timeliness) at all levels of quality (1 to 5). This task is difficult and time 
consuming and would require months of effort to obtain enough accurate 
information to create the utility function. Hence, to simplify the model further, 

I assumed that each division i produces its output j at a single quality level Sy. 
With this assumption, the objective function FfZy, Vi) can be written as 
F(Z jj ,S ij Vi,j), where Zy represents the quantity and Sy represents the quality 

of an output. 

2.3.3 Additivity of Objective Function 

It is obvious that if the amount and quality of every output of every division 
are maximized, all customers will be happy. But more than likely this strategy 
is cost prohibitive and not achievable under the existing resources. Therefore, 
trade-offs have to be made in which the merits of the outputs are compared. 
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I based this comparison on the impacts of outputs on the customers’ 
satisfaction, which were introduced into the objective function as weights for the 
outputs. 

Let Wjj be the weight that is assigned to the output X r I assumed that the 
worth of each output does not depend on the levels of other outputs of the 
division. For example, in the Facilities Planning Office (FPO), the happiness 
of the customers with the construction-of-facilities budget request is assumed 
to be unrelated to their happiness with the energy management output of the 
office. This assumption simplifies the objective function and is a reasonable 
approximation for a first-cut model such as this one. In particular, I assumed 
that total customer satisfaction can be estimated by adding across divisions the 
customers’ happiness with each individual division, which is a function of quality 
Sj, and quantity Z r With these assumptions, the objective function becomes 

F(Z,.S, Vij) = E,W,F s (Z,,S,) (2-7) 

In equation (2-7), the attributes are assumed to be additive independent. 
Keeney and Raiffa [1 976] described a three-step technique for verifying additive 
independence: (1) verify a property called utility independence; (2) verify a 
property called preferential independence; (3) if the previous properties are 
satisfied, determine if an additive or multiplicative functional form is appropriate. 
According to Keeney and Raiffa [1976], the attributes F ir and F r( (where i" * i' 
and j" ± j') are additive independent if the paired preference comparison of any 
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two lotteries defined by two joint probability distributions on F depends only 
on their marginal probability distribution. I have not performed the above steps 
owing to lack of time; however, I now describe how they might be undertaken. 

To verify preferential independence and utility independence, the set of 
attributes F y is partitioned into two subsets F ir and F rr . Each subset, in 
general, may have two or more attributes. First, utility independence between 
F iT and F iT could be verified by following an analogous procedure. The F iT is 
kept fixed at fj T . Then, a comparison between a 50-50 lottery and either 
another 50-50 lottery or a single certainty consequence is performed. The 
consequences in the paired comparisons are described in the F ir attribute 
space while the Fj-j- attributes are held fixed at Next, the F^» is changed 
to another level fjlp and the process is repeated for several values of f| T 
covering the range of F rr If the decision-maker’s preference between values 
of F ir does not depend on the value of f| T , it can be concluded that F iT is utility 
independent of F ir [Keeney and Raiffa 1976]. In practice, this is normally done 
by defining F rj » as a single attribute X rj .. The comparison are usually of the form 
shown in Figure 2.2. Here X? r is the certainty equivalent, X° T is the worst 
outcome, and X' T is the best possible outcome. If selecting the certainty 
equivalent X- T does not depend on fj r , then X ir is utility independent of F rr 
This procedure is repeated for all X iT . 
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Second, to verify whether partitioned-valued F r( is preferentially independent 
of F it , an f“ r is chosen at an undesirable level of attributes. Then, distinct 
values fj r and f' r are chosen such that the decision-maker is indifferent 
between (f| T , f; T ) and (fj' T , fj r ). The f: r is changed to a desirable level; if the 
decision-maker remains indifferent, fj r is preferentially independent of C. 
Usually, f iT involves just two attributes. This procedure is repeated n - 1 times 
for f iT = (Xjj, X iT ) for all i'j' ij when arbitrary ij is used as the basis of 
comparison. 

Third, to determine if an additive or multiplicative functional form is 
appropriate, a lottery is constructed as shown in Figure 2.3, 
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Figure 2.3 Additive and Multiplicative Comparison. 
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where in choice A all the elements are at either their best or worst values and 
in choice B some of the elements p are their best while the others are at their 
worst and vice versa. If the decision-maker is indifferent between A and B, the 
function is additive; otherwise it is multiplicative. 

2.3.4 Quality of Output Versus Quality of Division 

As defined earlier, quality is measured by assessing customer satisfaction. 
In particular, S, registers the customers’ opinions of the quality of output j of 
division i. To determine Sy, inquiries should be made on all the dimensions of 
quality (e.g., timeliness and communication; see Section 4.3) for all the outputs. 
For example, if an organization produces 50 outputs and 5 quality dimensions 
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are considered, a total of 250 data points should be sought from customers in 
order to determine the Sy for the Xy produced during, say, one year. The 
enormity of this task for the investigator and for customers may result in poor 
data (see Section 4.4). 

Furthermore, additional data are required to determine how resource 
allocation changes affect Sy. To reduce the size of the problem, the quality of 
outputs is assumed to be the same for the entire division rather than for 
individual outputs. In this scheme, if the organization has 6 divisions, 
customers are asked for their opinions on the 6 divisions and 5 quality 
dimensions rather than on the 50 outputs and 5 quality dimensions. This 
simplification eliminates the need for the index j for Si but may not be 
appropriate in an actual application and is made here just for convenience. 

To determine an indicator of satisfaction with the quantity of a division’s 
output, it is necessary to define an aggregate measure of an output by 
normalizing, weighing, and then adding all outputs of the division. Outputs Zy 
are normalized by dividing their value by their maximum value UZy (maximum 
feasible value). 

A function for F ij (Z ij ,S l ) should be selected to characterize the reasonable 
assumption that satisfaction is zero if quantity is zero. A simple function that 
represents this characteristic is the multiplicative form, which was selected 
here. Therefore, the satisfaction of customers with a division can be estimated 
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by 

Max EjW^U^)] (2-8) 

Considering equation (2-8), the objective function’s rate of change with respect 
to any variable is constant, given values of all other variables. For example, 
given Sj, the partial derivative of the objective with respect to \ is SiW/UX^, 
which is constant. 

2.3.5 Customers As Only Recipients of Each Division’s Output 

As a further simplification, I assumed that all outputs produced by a division 
are received by customers and are not used as inputs to other divisions. 

Future research should allow outputs of one division to be inputs to 
another. Some of the TSD products are actually combinations of outputs from 
several non-TSD divisions. But an assumption can be made that the 
customers are the recipients of all the outputs. If the outputs of two or more 
divisions are needed to complete a product, a customer could receive an output 
from one division and give it to another one to complete the product and so on. 
Caution must be taken in evaluating the final results of the model to ensure 
that the outputs that are required to complete the above product are available. 

To clarify this point, imagine that the customer would like to bake a cake. 
The customer receives eggs from one of the divisions, flour from another one, 
and any other ingredients from other divisions. Finally, the customer gives all 
the necessary ingredients to a division in TSD to bake a cake. It is essential 



30 


to ensure that all the ingredients, in the correct amount, are available to 
produce the cake. 

This assumption allows us to eliminate variables Y mp and Z (j by setting 
Zjj = X (j . With the above assumptions the following mathematical model is 
presented: 

Max Z = EiSi EjW^/UX, 

subject to 

Z,RS,(S,) + < b, 

s-ss.ss™ Vi 
LXy < Xjj < UXjj Vi,j 
and all variables are nonnegative. 

The decision variables are 
S, General opinion of customers on the quality of division i outputs on a 
1-to-5 scale 

Xjj Units of output j produced by division i 
The parameters and functions are 
B k Total resource of type k available to the organization 

LXjj Lower bound of X s 
RS jk (Sj) Use of resource k as a function of Sj 
RXjj k (Xy) Use of resource k as a function of X, 

Upper bound of Sj 
S min Lower bound of Sj 


(2-9) 

Vk (2-10) 

(2-11) 

(2-12) 
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UXjj Upper bound of 

Wjj Weight of output j of division i 

This model in its final format is presented in Section 6.4. 



PART TWO 


MODEL ESTIMATION 

The second phase of modeling is assigning numerical values to the 
variables. These values are determined by examining the historical data, 
implementing engineering analysis, or using expert judgments that depend on 
the managers of the organization and the needs and attitudes of the 
customers. Next, procedures are presented to estimate the parameters of the 
model. Although the procedures are the same for various organizations, the 
numerical values of the parameters depend on the type of organization and its 
structure, goals, and mission. 

Therefore, before estimating any parameters, a particular problem setting 
must be chosen. These methods are specifically applied to TSD to illustrate 
and further explain the procedures. Chapter 3 introduces the TSD, and 
Chapters 4 through 8 discuss the estimation of various parameters. 
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Chapter 3 

Modeling Technical Services Directorate 


In this chapter, the organization used as a case study in this thesis is 
defined, and the mission of the organization and its makeup are identified. This 
information is necessary for estimating the model parameters and executing the 
model in the following chapters. In Section 3.1 , the mission of the NASA Lewis 
Research Center is described, and in Section 3.2, TSD and its mission and 
goals are explained. In Section 3.3, the execution of the TSD model is 
discussed, and in Section 3.4, the process of data collection is explained. 

3.1 NASA Lewis Research Center Mission 

Below is a quote from the NASA strategic plan for the 1990s [Challenging 
the Future 1992]. This important information is the basis for the TSD mission, 
which is explained in following section: 

Our mission is to satisfy national needs through research, technology 
development, and systems development for aeronautical and space 
applications. We specialize in aero-propulsion, space propulsion, space power, 
certain aspects of space science and applications, and the related critical 
disciplines. NASA has designated Lewis as a Center of Excellence in the 
areas of aeronautical propulsion, space power, space communications, space 
nuclear propulsion systems, and space electric propulsion systems. This 
designation assigns the responsibility for producing research and technology 
advances in the given areas and for providing extended programmatic 
leadership, from research and technology to system development. Inherent in 
any Center-of-Excellence assignment is the requirement to form partnerships 
with participating centers and others having the expertise needed to contribute 
to the specific endeavor. Our role in the space chemical propulsion area is an 
example of such a partnership. In this area, we are a participating center. As 
such, we are expected to contribute both basic and focused technology in close 
collaboration with Marshall Space Flight Center (for large chemical systems) 
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and with Johnson Space Center (for small chemical systems). These are the 
designated Centers of Excellence and future developers of the flight systems 
that will require this technology. We are NASA’s designated Center of 
Excellence in the microgravity science disciplines of fluid physics, combustion 
science, and materials science. In the space science program, this designation 
assigns responsibilities that include the "principal investigator" function, 
experiment design, development, and operation, and publication of results. 
Finally, we are also responsible for delivering intermediate and large class 
expendable launch vehicle services for assigned missions. 

3.2 Technical Services Directorate (TSD) 

The organization under consideration, TSD, is managed by a director who 
leads the activities of four divisions and two offices, which will be referred to as 
a total of six divisions. Each division is headed by a division chief and is 
divided into several branches, the number depending on its size, that are 
controlled by branch chiefs. The work force in each branch reports directly to 
the branch chief, who reports to his or her division chief, who reports to the 
director. TSD employs over 700 civil servants and has an annual operating 
budget of approximately $50 million. 

The director of TSD makes decisions on the amount of resources allocated 
and the responsibility assigned to each division. Divisions are competing for 
eight different categories of resources, which are considered the inputs of the 
divisions and will be explained in later sections. The output of TSD is to fulfill 
its mission in the best way possible. 

Each division plays a specific role in meeting the directorate’s 
responsibilities and has a set of unique constraints that are not shared by other 
divisions. The only definite linkage among all of the directorate activities is that 
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the total resources of the divisions, manpower or funds, must not exceed the 
directorate budget allocation. They may also be outputs of some divisions that 
other divisions require as input. The mathematical formulation of the model 
can best be determined by examining the physical nature of the problem. In 
the Table 3.1, the division’s name, organization code, and index in the 
mathematical model are listed. The organizations and some examples of their 
outputs are then described briefly. 


Table 3.1 Divisions of Technical Services Directorate 


NAME 

ORG. 

CODE 

INDEX "i" 

FACILITIES PLANNING OFFICE (FPO) 

7010 

1 

OFFICE OF ENVIRONMENTAL PROGRAMS 
(OEP) 

7020 

2 

TEST INSTALLATIONS DIVISION (TID) 

7200 

3 

FACILITIES OPERATIONS DIVISION (FOD) 

7300 

4 

FABRICATION SUPPORT DIVISION (FSD) 

7400 

5 

FACILITIES ENGINEERING DIVISION (FED) 

7600 

6 


The Facilities Planning Office (FPO) provides the focal point for strategic 
planning of the construction-of-facilities activities to meet the Center’s 
programmatic and institutional facility needs. FPO manages and coordinates 
the advocacy of the Center’s Construction-of-Facilities (CoF) Program for 
rehabilitation, modification, or construction of research, research support, and 
institutional facilities. The division performs or directs studies of specific plans 
and support capabilities, including analyzing various courses of action and 
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recommending a preferred approach to management for new or improved 
research facilities, maintenance of the integrity of the physical plant, or 
improvements in productivity of the research facilities. FPO manages the 
Center’s Energy Conservation Program and serves as staff to the Facilities 
Review Board. FPO recommends facility plans that integrate requirements 
from all users. The division coordinates planning for non-CoF and 
maintenance-funded facility requirements. 

FPO manages the facility utilization and space management programs to 
ensure the most efficient use of the Center’s buildings and technical facilities. 
It allocates building space to meet research and institutional needs, manages 
the Center’s off-site office leasing activities, and prepares and maintains the 
Center’s facility utilization and real property reports. 

FPO also manages the facilities operations and testing support services 
contract to provide contract personnel for support to research, fabrication, and 
operations. 

The Office of Environmental Programs (OEP) is responsible for providing 
guidance and support to Lewis in five areas: industrial hygiene, environmental 
compliance, hazardous chemicals, health physics, and chemical sampling and 
analysis. OEP serves as a consultant to the Center staff in these areas. 
Occupational health and environmental hazards associated with the use of 
chemicals, radioactive materials, potable water, industrial waste, air and water 
pollution, and hazardous waste disposal are evaluated in accordance with all 
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pertinent National Research Council (NRC), Environmental Protection Agency 
(EPA), and Occupational Safety and Health Administration (OSHA) regulations. 
OEP implements policies that have been formulated by the Environmental 
Pollution Control Board. Area safety committees are apprised of 
environmental, industrial hygiene, and radiation issues through the safety 
permit review process. Medical surveillance programs are developed in 
coordination with the medical officer. In addition to supporting the office, 
analytical chemistry assistance is provided to the Center’s research programs 
and process systems technical support groups. 

The Test Installations Division (TID) comprises eight branches and 
provides mechanical, electrical, and electronic support services necessary for 
the installation, maintenance, modification, operations, and repair of the 
Center’s research apparatus and test facilities. These facilities include 
supersonic, subsonic, and icing research wind tunnels, high-altitude test 
chambers, space simulation chambers, zero-gravity drop facilities, shaker 
facilities, environmental test equipment, rocket and air-breathing engine test 
stands, and an aircraft that is used as flying test bed. The same services are 
supplied for the support of research on materials (composites, refractories, 
polymers, superalloys, etc.), electric propulsion, space power systems, and 
many other fields related to propulsion and power. 

The Facilities Operations Division (FOD) provides engineering and 
technical support for operation, modification, maintenance, installation, and 
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repair of research and institutional facilities throughout the Center. FOD 
schedules and operates the large prime movers (exhausters and compressors) 
of the central process air systems, supersonic tunnel drives, and the high- 
voltage electrical distribution network. The division manages the contracts to 
provide engineering and trade skills (millwrights, welders, etc.), research 
installation, major equipment, and systems maintenance. 

Services also include maintenance and repair of the central research 
systems, utilities systems, buildings, roads, and grounds. FOD manages 
technical services and maintenance support service contracts and provides 
functional management of the Center’s maintenance, custodial, buildings and 
grounds maintenance, and utilities (including heating, ventilating, and air- 
conditioning (HVAC)) budgets. FOD provides 24-hour services in support of 
fire protection and system alarms and administers a comprehensive security 
program concerned with identifying the need for and developing, implementing, 
and maintaining procedural and physical means for the protection and security 
of personnel, information, and property. FOD develops energy forecasts that 
provide technical and engineering support for the operation, modification, 
maintenance, installation, and repair of research support facilities, institutional 
facilities, and various systems. The division’s outputs are divided into 
institutional and R&D categories. The institutional products are facilities 
maintenance, ground maintenance, custodial services, security, and fire 
protection. The products supporting R&D activities are control and operation 
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of combustion air, exhaust air, atmospheric air, the variable-frequency drive, 
wind tunnels, and cooling towers. 

The Fabrication Support Division (FSD) provides information and 
hardware required by engineering, research, and service organizations at the 
Center. FSD evaluates, develops, and applies advanced technology in 
metallurgy, metal forming and joining, machining, wood model-making, 
instrumenting, inspecting, and nondestructive testing. These efforts cover wind 
tunnel models, aircraft modifications, cryogenic components, numerous test 
mockups, and spaceflight assemblies. FSD provides support and controls the 
application of equipment and technologies required to fabricate, instrument, and 
inspect hardware procured through outside sources. It provides metallurgical 
consultation and material selection services to engineering, technical services, 
and research divisions. 

The Facilities Engineering Division (FED) serves as engineering authority 
for construction of research and institutional facilities and systems. It 
establishes and maintains design and construction standards and drawings and 
records of facilities. FED analyzes designs of facilities, estimates costs and 
schedules, and provides construction inspection and contract management for 
facilities projects. It provides systems engineering and management, including 
operational and maintenance procedures, safety considerations, and systems 
configuration control and documentation for large, complex, centralized 
systems. FED also provides safety expertise through the Recertification of 
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Pressurized Systems Programs and the Building Life Safety Program and 
participates on Lewis safety committees and advisory panels. 

At this point, specific information is required to formulate a mathematical 
model of the directorate. The next few paragraphs are dedicated to 
establishing the mission of the directorate and the inputs and outputs, goals, 
and limitations of each division. 

TSD’s mission is to provide technical services to support both 
experimental research and facilities. The experimental research support 
encompasses research hardware, fabrication, instrumentation, installation, and 
appropriate test support. Facility support includes construction-of-facilities 
planning, design, construction, operation, and maintenance and covers both 
research and institutional facilities. TSD plans for and directs engineering, 
technical, and trades personnel involved in providing functional support and 
provides the necessary facility coordination with other centers and appropriate 
government agencies. 

TSD also directs the Center’s environmental compliance, industrial hygiene, 
and energy management programs. In addition, TSD has overall responsibility 
for all construction performed for the Center. TSD fabricates the R&D 
hardware and installs the test rigs and necessary equipment [NASA 
Organization Manual 1990]. 

TSD’s goal is to advance the Lewis mission by providing the best services 
possible to the other directorates at the Center. The objective of TSD, as it 
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was developed through interviewing the division chiefs, is to become the best 
provider of services to other directorates. This means that the directorate 
should strive to maximize its customer satisfaction. TSD’s customers could be 
internal or the other eight organizations that make up the Center. These 
directorates have specific functions and their efforts are concentrated and 
focused to satisfy the Lewis mission. TSD decision-makers should formulate 
quality improvement policies based on increasing customer satisfaction. TSD 
should anticipate customers’ problems and seek solutions as if they were its 
own problems. This objective can be met only by focusing on customers 
requirements and becoming a full partner in securing their goals. 

3.3 TSD Model Implementation 

For this study, a model for optimum resource allocation was applied to TSD 
to demonstrate the concept and identify the pitfalls and limitations of the 
procedure. The first task was to introduce the procedure to the decision- 
makers and seek their support. The best way of obtaining backing for any 
project in a hierarchical organization is to begin from the top of the hierarchy, 
where the director of the organization resides. To increase the chance of 
success in convincing the director on the merits of the process, the approval 
of my division chief, one of the decision-makers, was needed and obtained. 

With the sponsorship of my division chief, a presentation of the resource 
allocation concept was made to the director of TSD and he accepted the 
project. With the director’s blessing, the process was introduced to all division 
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chiefs and implementation of the optimum resource allocation was instituted. 
The remainder of this chapter documents various procedures I employed in 
gathering information to gain insight into the operation of the organization and 
tailor the mathematical model to the directorate. 

The mathematical model for TSD is based on the model developed in 
Chapter 2, where parameters and decision variables are uniquely developed 
for the directorate under consideration. Accurate development of the 
parameters requires specific information. A data collection process was 
developed and executed for gathering detailed information. These data are 
representative of the TSD mission, current resource allocation at the division 
level, and decision-makers’ visions for the future of the directorate. Specific 
parameter estimation procedures are the subject of Chapters 4, 5, and 6. 

3.4 TSD Internal Data Collection 

Data gathering, in general, was accomplished by asking knowledgeable 
Lewis personnel either through face-to-face interviews or by questionnaire. 
Both techniques were used, as appropriate, in different parts of this study. The 
choice of data collection techniques and the types of questions depend on the 
type of data desired, the ease of implementation of one technique over the 
other, the number of informants, and the level of accuracy required. 

After considering these criteria, the interviewing method was selected for 
obtaining information from decision-makers, and the questionnaire method was 
chosen for gathering information from customers, as explained in Chapter 4. 
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The group of decision-makers in this study consisted of the director of TSD and 
the six division chiefs, referred to as "the group" hereafter. 

The interviewing method was used to capture the thought processes of the 
decision-makers and their opinions on alternatives, attributes, and priorities. 
The reasons for this selection are as follows: (1) the group of decision-makers 
was small, only seven people; (2) a reasonable amount of time was available 
for interviewing; (3) the margin of error in interviewing is much smaller than in 
the questionnaire method. 

An interview is a flexible and interactive technique that produces up-to-date 
data. A direct contact in a personal interview enables the investigator to clarify 
any misinterpretation about the question. An interview can be structured or 
unstructured, formal or informal. According to Howard and Peters [1990], these 
types of interviews should be conducted in a structured and formal manner. 
At this stage, the researcher is faced with the problem of what questions to ask 
and in what format. 

Two types of questions may be asked: an open question and a closed 
question. An open question is used when the researcher wants the interviewee 
to give his or her opinion on the subject and elaborate on the answer. A 
closed question, on the other hand, suggests an answer from predetermined 
alternatives and limits the response of the informant [Neter et al. 1978]. 

Before starting the dialog with the division chiefs and beginning to solicit 
information, it was essential to determine the expected outcome of the 
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interviews. Therefore, a process for conducting the interviews and choosing 
the type of questions was developed and discussed with the director for his 
approval. The inputs from the director are boldfaced in the following process. 
The purpose of this process is to 

- ESTABLISH THE OBJECTIVES OF THE DIRECTORATE. 

- ESTABLISH THE DIVISIONS’ FUNCTIONS. 

- ESTABLISH THE DIVISIONS’ RESOURCES. 

- DEFINE THE OUTPUT OF THE DIVISIONS QUALITATIVELY. 

- ESTABLISH THE MEASURING UNITS FOR THE OUTPUT. 

The divisions’ functions and resources were determined by taking an 
inventory of all their activities, which constitute over 150 functions, and their 
associated resources. The results of this investigation were presented to the 
group for accuracy and concurrence (see Appendix D). At this point, many 
functions have to be evaluated, a challenging and time-consuming task for the 
busy decision-makers. 

One way to group these functions and reduce the size of this problem is to 
deal with the outputs of a division rather than its functions. For example, 
functions such as drafting and engineering have an output, which is a design. 
The product of several functions is classified as an output. The outputs that 
were identified, the resources that create the outputs, and specific questions 
that were asked to obtain this information are presented in Chapter 5. 


Chapter 4 

Present Quality of Division 

In this and the next three chapters, I discuss the specific procedures used 
to estimate the parameters of the model. In this chapter the present quality of 
a division BS, is determined. These data are essential for estimating the 
resource use function for the quality RS ik (Sj) of equation (2-10), which is 
discussed in Chapter 6. 

In Section 4.1, the quality of output is defined, and in Section 4.2, the 
method of measuring the quality by using questionnaires is presented. The 
questionnaires are based on several quality dimensions that are explained in 
Section 4.3. In Section 4.4, creating an questionnaire that measures the 
quality of output is discussed, and in Section 4.5, the methods that examine the 
reliability and validity of a questionnaire are presented. In Section 4.6, the 
concepts introduced in this chapter are applied to TSD for illustration. 

4.1 Quality of Output 

The organization’s production as measured through the quantifiable output 
may not give a true picture of its value if the quality of the output is not 
quantified. The link between quality of output and the success of an 
organization in private industry is well known and accepted. A study of more 
than 2500 businesses found that organizations with large market share and 
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higher quality output earn profit margins five times greater than organizations 
with smaller market share and lower quality output [Buzzell and Gale 1 987]. 

Steingraber [1991] reports that research by A.T. Kearney Inc. reveals that 
chief executive officers consider quality as a prerequisite for survival of their 
companies. Quality is defined as the extent to which the product meets the 
user’s expectations [Muller 1991, Lewis and Mitchel 1990, Marr 1986, 
Montgomery 1985]. According to Strozier [1991], the goals of many successful 
service companies are defined in terms of results achieved for customers 
rather than service performance. This definition confirms the idea that the best 
judges of quality of output are the customers who receive the service. 

In addition, government agencies are challenged to implement total quality 
management (TQM) principles, which require that organizations continue to 
improve the quality of their outputs. Hyde [1991] articulates that TQM relies 
heavily on direct feedback from recipients of products and services for 
assessment of quality. The best measure of quality is quantitative feedback 
from customers. This conclusion is recognized and successfully implemented 
by companies such as Xerox and IBM [Marr 1986]. 

It is also important to note that services usually go unnoticed when the 
customers are satisfied. Only when the customers experience problems do 
they pay attention to the quality of service [Levitt 1984, Leonard 1987]. This 
situation requires that organizations not only aim to maximize the service 
quality, but also be prepared to quickly eliminate any problems that are detected. 
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The definition of quality varies from company to company and from output 
to output. There are so many definitions that they are classified in five 
categories: transcendent, product based, manufacturing based, value based, 
and user based [Garvin 1988]. I explain each category in the following 
paragraphs. 

The transcendent definition of quality is generally a level of excellence that 
is based on personal feeling [Tuchman 1980]. In other words, quality is a 
characteristic of a product or service that we know when we see it. The 
problem with this definition of quality is that it is vague and without practical 
guidance. 

Product-based quality is defined as the amount of certain ingredients or 
attributes present in the product [Abbott 1 955]. For example, a higher quality 
Persian carpet has more fiber knots per square inch. This approach associates 
the quality of a product with the amount of the desired attribute it possesses. 
There are two problems with this definition: (1) higher quality is achieved 
through higher cost and (2) quality is not acquired but assigned to a product 
on the basis of its component. In the case of TSD, this means that if the 
organization employs highly educated engineers, it can be assumed that they 
possess attributes such as communication, responsiveness, and timeliness and 
any other quality aspects of a service organization. This assumption is 
obviously not valid. 
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Manufacturing-based quality is basically conformance of the product 
specification to preestablished characteristics [Crosby 1979]. Under this 
definition, if an automobile is manufactured to its exact specification, it is a 
high-quality car regardless of the requirements. This means that a Cadillac 
made to its specification has the same quality as a Chevette made to its 
specification. This definition of quality may be appropriate for some of the 
divisions in TSD, such as Fabrication Support Division (FSD) and the Test 
Installations Division (TID), that have fabrication capability. As long as 
customers define the specification of the product and the part meets or 
exceeds those requirements, the divisions will have satisfied customers. 

Value-based quality is defined in terms of cost and price. A product that 
performs to its specification is a quality product if it is offered at a reasonable 
price. Therefore, a $100,000 well-made and well-engineered car may not be 
a quality product because its price may not be reasonable [Broh 1982]. 
Although this definition of quality is becoming more and more popular, it is not 
suitable for this research. 

User-based quality is directly related to the attitude of consumers toward 
the product. This concept seeks the exact combination of the product 
attributes that will satisfy the most customers. The more satisfied the users 
are, the higher is the quality of the product [Edwards 1968]. For example, in 
a service organization such as TSD, the quality of output is directly related to 
the level of customer satisfaction. The best judge of this type of quality are the 
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customers who are the recipients of the service. Coppola [1991] noted that 
quality is ultimately defined by customers, and he warned against self- 
evaluation of quality, which can be disastrous for the organization. The user- 
based definition of quality is used in this thesis because its philosophy is 
parallel with the concept of TQM, which is used in the organization. 

Because I defined quality as the degree of customer satisfaction, the next 
section is devoted to developing a system for measuring customer satisfaction. 
4.2 Measuring Customer Satisfaction 

The organization must identify the needs of customers and take appropriate 
actions to meet or exceed their needs and consequently maximize customer 
satisfaction. Customer satisfaction should be defined and measured 
quantitatively in order to be maximized. Customer satisfaction is defined as 
meeting customer requirements on quantity and quality of service and product. 
The measurement, in general, is assigning numbers to objects or events 
according to rules [Stevens 1951]. This definition is suitable for measuring 
tangibles, such as the number of widgets or drawings. For measuring 
intangibles, Blalock [1974] suggested that measurement can be viewed as a 
process of linking abstract concepts to empirical indicators. Because the 
quantity of the service or product is tangible, it can be determined by simple 
questions on the level of output. On the other hand, the quality of output is not 
easily articulated by the customer, and it is difficult to measure. 
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In this study, the attitudes of customers were measured by directly asking 
them about the quality of the organization’s outputs. Specifically, customers 
were presented with a list of statements about the specific services provided 
by the organization and asked to agree or disagree with each statement. This 
procedure, which Nunnally [1978] called attitude scales, was conducted to 
determine how satisfied the customers are with the services provided by the 
organization. First, the dimensions of quality had to be identified, and then 
each had to be measured. 

4.3 Identifying Quality Dimensions 

Quality is a multidimensional entity. Garvin [1988] proposed to break it 
down into several dimensions, known as quality dimensions. These 
dimensions are representative of how the quality of a product or service is 
judged by customers. In a way, quality dimensions are customer satisfaction 
dimensions. There are two methods for developing quality dimensions: the 
quality dimension development approach and the critical incident approach 
[Hayes 1 992]. Quality dimensions are developed specifically for each division 
even though some dimensions may be suitable for all the divisions. 

The critical incident approach asks every customer to identify specific 
performance examples that illustrate the customer’s perception of the quality 
[Flanagan 1954]. A critical incident is a specific example that the customer 
provides to demonstrate a positive or negative aspect of a product or service. 
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In examining many customers, the critical incident approach would not be a 
practical concept. 

Quality dimension development involves the people who provide the service 
and understand its purpose and function [Hayes 1992]. These experts 
establish a set of quality dimensions that mostly pertain to their organization. 
For instance, in the case of TSD, experts are the division chiefs, who are most 
familiar with their products and can establish quality dimensions of their 
outputs. For example, the chief of the Test Installations Division (TID) selected 
6 quality dimensions from a list of 21: availability, communication, 

responsiveness, reliability, flexibility, and competence. A list of the 21 quality 
dimensions with a brief explanation of each is given in Appendix A. 

4.4 Measuring Quality 

At the point where quality dimensions are identified, a measurement 
instrument should be developed to determine the attitude of the customers 
toward these quality dimensions. According to McNeal and Lamb [1979], 
customer satisfaction is most typically measured through surveys. A survey is 
a series of statements that are designed to gather specific data for developing 
policies or determining a course of action [Oppenheim 1966]. 

Although the contents of the surveys differ, the procedure for creating them 
is the same. The main point that must be considered in designing surveys is 
that completing a questionnaire is an imposition. Therefore, the number of 
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questions should be kept to a minimum by requiring only precise information 
related to the objectives of the study. 

The quality dimensions are the bases for designing questionnaires for 
customer satisfaction. The content of the questionnaire has certain 
characteristics: (1) the questions should be relevant to the objectives of the 
questionnaire; (2) the questions should be concise; (3) the questions should be 
unambiguous; (4) the questions should contain only one thought; (5) the 
questions should not contain double negatives. 

The next important step after developing questions is selecting a procedure 
to quantify the response of the customers. Several response formats or scaling 
methods that are used to assign numbers to the customers’ attitudes were 
considered: Thurstone’s method of equal-appearing intervals [Thurstone 1 927], 
Likert’s scaling technique [Likert 1932], Guttman’s scalogram approach 
[Guttman 1950], and item response theory (IRT) [Lord 1980]. 

The Likert scale was implemented here because of its simplicity and high 
reliability [Edwards and Kenney 1946]. Likert [1932] introduced a summative 
scale that is typically used to scale people with respect to their attitudes. The 
method asks customers to rate an attribute from excellent to unsatisfactory or 
from approved to disapproved or from strongly agree to strongly disagree by 
assigning a numerical value to their attitudes from a predetermined range. For 
example, a customer may be asked to evaluate the response time of 
maintenance calls from a continuous range of 1 to 5, where 1 represents 
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unsatisfactory and 5 represents excellent (see Appendix E for various 
questionnaires). As Nunnally [1978] pointed out: 

Likert scales have a number of attractive advantages over all other methods: 
they (1) follow from an appealing model, (2) are rather easy to construct, (3) 
usually are highly reliable, (4) can be adapted to the measurement of many 
different kinds of attitudes, and (5) have produced meaningful results in many 
studies to date. 

The quality dimensions and the method of measuring customer satisfaction 
have now been explained. In the next two sections the reliability and validity 
of this measurement technique are examined. 

4.5 Reliability and Validity of Measurement 

Like any type of measurement, there is error in measuring customer 
satisfaction by methods such as the Likert scale. Measurement error can be 
in the form of systematic error S or random error R. When a customer is 
asked about his or her opinion on a specific quality dimension, the answer is 
the level of the customer satisfaction with the product or service, which is 
known as a score for the question. However, this is not a true score T of 
satisfaction but an observed score X. In other words, there is error involved 
that distorts the observed scores away from the true scores [Carmines and 
Zeller 1979, Lord and Novick 1968, Nunnally 1978]. The source of error may 
be attributed to the following factors [Selltiz et al. 1976]: (1) the person’s 
willingness to express his or her true feeling, (2) the person’s mood, (3) the 
method of acquiring information, (4) the method of administering the interview 
or questionnaire, (5) the wording of the questionnaire, (6) an ambiguous 
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statement in the questionnaire, (7) mechanical factors, such as circling the 
wrong number. Symbolically the relationship is 

X = T + S + R (4-1) 

Two important issues must be addressed in any measurement: reliability 
and validity. Kerlinger [1 973] equated reliability with dependability, stability, 
consistency, and predictability. Nunnally [1978] defined reliability as the extent 
to which measurements are repeatable. Reliability is directly related to the 
influence of random error on the measurement. If R = 0, the measure is 
perfectly reliable. 

Validity is the degree to which the scale measures what it is supposed to 
measure. Referring to equation (4-1), a measurement is valid when X = T. 
There are three types of validity: content validity, criterion validity, and 

construct validity [Golden et al. 1984], which I discuss later in this chapter. 
Next, I review methods for assessing a questionnaire’s reliability and validity. 
4.5.1 Reliability 

The reliability tests are based on the concept of correlation. Correlation is 
the strength of a relationship between two things. A correlation coefficient is 
the numerical index that expresses the linear relationship between two 
variables [Cohen et al. 1 988]. The correlation coefficient ranges from -1 to +1 . 
A coefficient of +1 or -1 implies a perfect linear relationship; a zero value for 
a correlation coefficient means no relationship between the two variables. A 
positive value for a correlation coefficient indicates that the two variables 
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simultaneously increase or decrease together. A negative correlation generally 
means that if one variable increases, the other one will decrease. 

There are four forms of reliability tests for a questionnaire: test and retest, 
alternative form, split-halves, and internal consistency. Test-retest is a 
reliability test that examines the correlation between the scores of the same 
questionnaire given twice to the same people. The alternative form method is 
similar to the test-retest method except that this technique searches for the 
correlation between the scores of two versions of the same questionnaire given 
to the same people. Nunnally [1 978] recommended that the two questionnaires 
in either method be administered about two weeks apart. These methods were 
not suitable here because the customers were not willing to respond to two 
questionnaires about the same subject within a two-week period. Also, 
generating alternative forms of the questionnaire that have the same 
characteristics is difficult. 

The split-halves reliability test follows the same concept as the alternative 
form reliability test with the exception that it is developed by dividing the 
questionnaire into two halves. It tests for the degree of consistency across 
items of the questionnaire. The advantage of this test is its single 
administration. The disadvantage is that the test does not estimate the stability 
of the score. However, its major problem is the issue of how to split the scale. 
Whatever method is chosen to split the scale (e.g., odd-even split), the split- 
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halves reliability test considers only one possible split rather than all the 
possible splits. This problem brings us to the last and final reliability test. 

The internal consistency reliability test, unlike the split-halves test, reveals 
the internal relationship among the items of a questionnaire. Two methods 
were considered for estimating the internal consistency reliability: the Kuder- 
Richardson 20 formula (KR20) and the coefficient alpha method. KR20 is 
designed to estimate the reliability of questionnaires with questions that have 
two possible answers, such as true-false. The coefficient alpha method is 
suitable for calculating the reliability of questionnaires with questions that have 
many possible answers [Cronbach 1 951 , Ebel 1 965, Kaiser and Michael 1 975]. 
In the coefficient alpha method, which was used in this study, the reliability of 
a questionnaire is calculated by using the correlation between questions that 
are based on one quality dimension, say availability, and questions that are 
based on another quality dimension, say timeliness. 

4.5.2 Validity 

Three methods are relevant for measuring the validity of a questionnaire: 
criterion validity, construct validity, and content validity. Criterion validity 
examines a questionnaire for the predictability of the behavior that the 
questionnaire is testing. In the case of customer satisfaction, this type of 
validity examines the relationship between quality dimensions and customer 
behavior. For example, if customers are satisfied with the service provided, 
what is the chance that they will increase their business with the company? 
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The test is valid if the correlation between the customer satisfaction test and 
increased business reaches an acceptable level previously identified by 
decision-makers [Golden et al. 1984]. 

Construct validity involves studying variables that should have a strong 
relationship with the test domain (customer satisfaction) and variables that 
should have no relationship with the domain. A test is valid if the scale 
correlates with the variables that it should (convergent validity) and does not 
correlate with the variables that it should not (discriminate validity) [Campbell 
and Fiske 1959]. For example, it is expected that customer satisfaction should 
be related to the timeliness of service. Also, there should be no connection 
between customer satisfaction and the color of customers’ hair. If a slight 
correlation is calculated between customer satisfaction and service timeliness 
or a strong correlation between customer satisfaction and the color of their hair, 
the test is invalid [Hayes 1992]. 

Content validity, which was used in this study, is the degree to which the 
items in the test are representative of all possible items that could be included 
in the customer satisfaction questionnaire. In other words, does the customer 
satisfaction questionnaire cover all the quality dimensions that it attempts to 
measure? Content validity is judged by the people who are most familiar with 
the purpose of the questionnaire comparing the content domain with the test 
items [Nunnally 1978]. Following this logic, a questionnaire is valid if it is 
inclusive of all the quality dimensions that the division chief decided to 
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measure. Therefore, to develop a valid questionnaire, decision-makers should 
be directly involved in creating their respective questionnaires and should 
approve the questionnaires to assure that all the quality dimensions are 
included. 

In summary, if a test is valid, the observed scale is equal to the true scale 
or X = T. Referring to equation (4-1), this means that the test is also reliable 
because R = 0. But if the measure is reliable (R = 0), it is not necessarily valid 
(X = S + T). Therefore, reliability is a necessary but not sufficient condition for 
validity [Churchill 1979]. The information on reliability and validity was 
considered in developing the questionnaires for TSD in order to produce valid 
and reliable data. In the following section the concept of developing the quality 
score for an organization is applied to TSD for illustration. 

4.6 Present Quality of Division BS, 

The quality of output, as described previously, was measured by assessing 
the customers’ general opinions on the quality of the division’s output in fiscal 
year 1992 (BSi) and was used as a data point to estimate the quality resource 
use function. This assessment was done through a questionnaire that was 
uniquely designed for each division. The variable Sj is an indicator of the 
general opinion of customers about division i for a given year and set of output. 
The higher the value of Sj, the higher is the quality of output. The lower bound 
S min and upper bound S max of the quality scale were arbitrarily selected to be 1 
and 5, respectively, according to the Likert scale. 
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By following the procedures presented in Section 4.3, a list of possible 
quality dimensions was identified through a literature search for the division 
chiefs’ consideration. This list contains 42 quality dimensions with a brief 
explanation of each dimension (see Appendix A). Some of the dimensions may 
be repeated and some may not be applicable to a specific division. After 
eliminating similar dimensions, the division chiefs considered the following 21 
items as proper quality dimensions for their divisions: reliability, 

responsiveness, competence, access, courtesy, communication, credibility, 
security, understanding, tangibles, performance, features, conformance, 
durability, aesthetics, convenience, completeness, timeliness, steering, 
flexibility, and follow-up. 

The questions used were developed by reviewing the literature [Nelson 
1978, Parasuraman et al. 1985, Amsden 1989, Armistead 1989, Kennedy and 
Young 1989, Hyde 1992, Hayes 1992] and a questionnaire from Xerox 
Corporation. Also, the questions were individually tailored to meet the 
divisions’ requirements and the division chiefs’ expectations. 

A preparatory questionnaire was developed for each division. To ensure 
validity, every division chief approved the questions and confirmed that the 
questionnaire contained all the selected quality dimensions. Next, the division 
chiefs selected a small group of customers that represented a cross-section of 
all the customers of the division. The preliminary questionnaire was presented 
to these sample customers for their input. After the responses to the 
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preliminary questionnaire were received, the questionnaire was revised, 
proceeded through the same approval cycle as the preparatory questionnaire, 
and was distributed to all the customers. 

The only division that did not follow this procedure was FED. This division 
did not accept the questionnaire that was designed specifically for them and did 
not send a preliminary questionnaire to a set of sample customers. FED 
designed its own questionnaire, which was vague and not very useful for this 
research. After several discussions with the division chief, an agreement was 
reached to combine both questionnaires into one and use it for all of their 
customers. 

The preliminary and final questionnaires for all the other divisions did not 
differ a great deal. Appendix E contains the final questionnaires, and in 
Section 7.8, the results for each division are presented. It is evident that the 
questionnaire is a diagnostic tool that points out the problems with the quality 
of the output. It is up to the division chiefs to take appropriate action to 
eliminate these problems. In the next section, I offer a method to help 
decision-makers allocate proper resources so as to improve the quality of 
division outputs. 

4.6.1 General Satisfaction Versus Quality Dimensions 

To increase the quality level S, decision-makers should strategically allocate 
scarce resources to reducing or eliminating any quality deficiency or use the 
resources available more efficiently. The challenge is to determine or estimate 
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where to concentrate the resources so as increase S f . The relationship 
between resources and quality (customer satisfaction) can be developed by 
executing several resource allocation cycles. Each cycle consists of conducting 
a survey to determine the level of customer satisfaction and then adjusting 
resources to improve customer satisfaction and recording the effect of the new 
resource allocation by conducting another survey after six months or a year. 

Because this was the first time that the cost of quality was considered by 
TSD, historical data were not available. Therefore, it will be impossible to 
predict the effect of various resource allocations on the quality of output from 
actual data. Developing the relationship between quality and resources by 
experimentation is virtually impossible because I would have to systematically 
change the resources that support the quality of outputs, which is customer 
satisfaction, and register the change in customer satisfaction by administering 
a new questionnaire. This process is time consuming and bombards the 
customers with various questionnaires that will have an adverse effect on their 
cooperation and the quality of data they provide. Therefore, in this model, the 
cost of quality will be estimated by the experts, the people who are closest to 
the operations. 

It is much easier to determine the cost for improving a specific quality 
dimension, such as timeliness or availability (defined in Section 4.3), than the 
cost for improving the general opinion of customers. Thus, to find the 
relationship between quality dimensions and current general satisfaction and 
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determine the quality dimensions that most influence the general opinion, a 
multiple regression analysis was performed between general satisfaction, as 
the dependent variable, and quality dimensions, as the independent variable. 
In other words, BS, was determined as a function of quality dimensions. Then, 
the cost of improving quality dimensions could be estimated by the division 
chief, and when the cost of improving each quality dimension was known, the 
overall cost of improving S could be estimated. The task of developing general 
satisfaction as a function of quality dimensions was accomplished through 
regression analysis, which is described next. 

4.6.2 Regression Analysis 

Regression analysis is a technique that models a dependent variable as a 
function of independent variables. The general form of this model for m 
observations and h variables is 

BS im = Pio + PiiQim1 + PkAiii2 + ••• + PilAmh + e im V m ( 4_2 ) 
where m is the total number of customers that responded to the division i 
questionnaire; e ifn is the deviation of the calculated value from the observed 
value of the dependent variable; BS im is the score that a customer of division i 
gave to the general satisfaction questionnaire in a base year, which is 
considered a dependent variable; and Q imh are scores for h quality dimensions 
of division i, which are considered independent variables. Quality dimensions, 
as defined in Section 4.3, are representative of how the quality of a product or 
service is evaluated. For a service organization, quality dimensions are 
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attributes such as timeliness and communication. The (3* coefficients are 
estimated by minimizing the sum of squares of deviations from the observed 
value of BS jm . 

E = Xi m e f m = 2i m (BSjm - (Jjo “ PiiQjml — Pi2^im2 “ — ~ PihQmh) (4‘3) 

A set of simultaneous equations resulting from equating the partial 
derivative of E with respect to each p jh to zero were generated. The h + 1 
simultaneous linear equations are most easily solved with a computer program. 
The STORM software was used to perform a regression analysis for TSD. The 
regression equations were developed by starting with null regression and 
performing a stepwise procedure to the end. 

The stepwise procedure decides from a large set of independent variables 
which one of the variables has the most influence on the dependent variable. 
In this screening procedure, the user first identifies the dependent variable and 
a number of independent variables. Then, the user performs a null regression 
in which the procedure creates a model (relation between independent and 
dependent variable) with no independent variable. In the next step, known as 
a forward step, the user enters an independent variable into the model if its 
significance level in an F-test is less than a preestablished value. Then, in a 
backward step, the user deletes the variables that are already in the model if 
under a t-test their significance value is greater than a preestablished value. 
To avoid cycling, STORM will not allow deleting a preestablished value that is 
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less than an entered preestablished value [Hamilton et al. 1992]. Detailed 
explanations of the stepwise procedure are presented is various statistics 
books [McClave and Dietrich 1979, Makridakis et al. 1978]. 

The equation is also validated as follows: part of the data is used to 
develop the regression equation; the equation can then be used to predict the 
dependent variable value for the remaining cases; and the accuracy of the 
prediction is determined. 

In this study, 60 percent of the data was used to develop the regression 
equation for the validation procedure. Also, for the sake of accuracy of the 
equation, the variables were entered and deleted from the equation to match 
the adjusted R 2 as close as possible to the validation R 2 , where R 2 is the 
degree of fitness of the equation to the data set The validation R 2 is the 
degree of fitness between the regression equation, which was developed with 
60 percent of the data, and the remaining 40 percent of the data. An 
experimental R 2 of 0.6 or higher is acceptable for this type of study. This claim 
can be proven by using the F-test, which allows testing the significance of the 
overall regression model. From Makridakis et al. [1978], the F value can be 
calculated by using 

F = [R 2 /p]/[(1 - R 2 )/(N - P - 1)] (4-4) 

where 

N Number of observations 
p Number of parameters in regression 
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For illustration purposes, equation (4-4) is applied to the Facilities Planning 
Office, where R 2 = 0.53, N = 25, and p = 2 (timeliness and responsiveness). 
Using appropriate tables, the F value based on p and N - p - 1 at 95-percent 
confidence is 8.65. Now, from equation (4-4), R 2 = 0.44. Because the FPO’s 
R 2 (0.53) is greater than the computed R 2 (0.44), there is a 95-percent 
confidence that the linear regression equation for FPO is representative of the 
data acquired on FPO. 

The customers’ general satisfaction as a function of other quality 
dimensions is listed below for each division. The quality dimensions are 
identified in parentheses next to the question number in the questionnaire: 
FPO: [Adjusted R 2 = 0.53] General Satisfaction = 1.6 + 

0.39(Timeliness #10) + 0.24(Responsiveness #9) 

OEP: [Adjusted R 2 = 0.39] General Satisfaction = 1 .96 + 
0.456(Convenience #3) + 0.1 (Convenience #4) 

TID: [Adjusted R 2 = 0.68] General Satisfaction = 0.4 + 

0.42(Communication #5) + 0.26( Availability #3) + 
0.186(Responsiveness #9c) + 0.1 (Demand #20d) 

FOD: [Adjusted R 2 = 0.74] General Satisfaction = 0.07 + 

0.56 (Communication #7) + 0.43(Convenience #3) 

FSD: [Adjusted R 2 = 0.54] General Satisfaction = 1.12 + 

0.41 (Communication #2) + 0.34(Communication #6) 
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FED: [Adjusted R 2 = 0.83] General Satisfaction = -0.1 + 

0.45 (Responsiveness #7d) + 0.27(Communication #5) + 

0.2 (Responsiveness #7b) 

The quality dimensions identified in this step and the comments that were 
solicited through questionnaires will give the decision-makers a basis for 
estimating the cost of maintaining or improving upon the current level of 
general satisfaction. For example, if the Facilities Operations Division needs 
to increase the general satisfaction, it has to concentrate its effort mostly in 
communication and then convenience because these two dimensions are most 
closely associated with general satisfaction. However, the relationship between 
quality dimensions and general satisfaction is only half of the story; the cost of 
improving quality dimensions is also needed. I discuss this in Chapter 6 by 
making general assumptions to assess the cost of quality. 


Chapter 5 

Estimation of Output Bounds and Weights 

In this chapter, the weight of each output as a part of the objective function, 
equation (2-9), is conceptually developed by using the analytic hierarchy 
process (AHP) and is then applied to TSD as an illustration. The represent 
the decision-makers’ opinions on the importance of the outputs to the 
customers’ general satisfaction. In Section 5.1 , 1 describe severed methods that 
I considered in this study and the reasons behind the selection of AHP. In 
Section 5.2, the AHP method is applied to TSD and weights for each division 
are calculated. In Section 5.3, the upper and lower levels of outputs are 
determined. 

5.1 Methods of Estimating Weights 

The weight for each output is the only means by which decision-makers can 
directly influence the objective function of the model. It is important to select 
a correct technique that is easy to implement and understand. There are 
several ways to develop the weight of each output. The easiest method is to 
assume equal weights for all outputs. Another method is to allocate 1 00 points 
among outputs in proportion to their importance. Unfortunately, direct methods 
often fail to yield weights that correspond to trade-offs which people are willing 
to make [Hobbs et al. 1992]. The reason for this may be a vague definition of 
"importance'' [Schoemaker and Waid 1982]. Another group of methods are 
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based on the multiattribute utility theory developed by Keeney and Raiffa 
[1976]. 

Utility is defined as assigning numbers that indicate how much something 
is valued [Berger 1985]. The concept of the multiattribute utility theory is based 
on constructing a multiattribute utility function, which requires developing a 
utility function and scaling factors for each attribute. The single-attribute utility 
function may be produced by using, for instance, a variable probability method 
[Von Winterfeld and Edwards 1986]. In a variable probability method, the 
decision-maker is presented with questions involving two options: (1 ) a gamble 
that has the probability P of the best possible outcome and the probability 
1 - P of the worst outcome and (2) an intermediate outcome that is considered 
a sure thing. 

The decision-maker is asked to specify a probability P such that he or she 
would be indifferent between the gamble or taking the intermediate outcome. 
Mathematically, if O* is the best possible outcome, O* is the worst possible 
outcome, and O is an intermediate outcome, the preference order of the 
outcomes is 

O* > O > O* (5-1) 

The probability that O occurs is P, and the probability the O* occurs is 1 - P. 
The question is at what value of P the decision-maker is indifferent between 
taking the gamble (O* and O*) or taking the sure thing (O). Once P is 
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identified, the utility of the best and worst outcomes is set to 1 and 0, 
respectively, and the utility of O is calculated by 

U(O) = PU(0*) + (1 - P)U(0*) (5-1) 

The weight is constructed by presenting to the decision-maker a gamble 
with the probability P(i) of winning the outcome with the best values for 
attributes and the probability 1 - P(i) of receiving the worst values for all 
attributes. This gamble is compared with a sure thing that has the worst values 
on all attributes but the i m one, where it has the best outcome. The decision- 
maker is asked to adjust the probability P(i) until, in his or her opinion, there is 
no difference between the gamble and the sure thing. Setting the utility of the 
best and worst outcomes to 1 and 0, respectively, the value of P(i) elicited from 
the decision-maker represents the weight of the i* 1 attribute in the multiattribute 
utility function [Keeney and Raiffa 1976]. 

Another class of methods is based on using ratio questioning for ranking 
the alternatives. The method requires information on the ratio of importance 
of two outputs at a time [Hobbs et al. 1992]. The analytic hierarchy process 
is a version of the ratio questioning method developed by Saaty [1977]. AHP 
is one of the easier methods to use and understand because it allows decision- 
makers to visually see the structuring of the decision process. 

AHP may be viewed as a procedure for scaling and weighting attributes and 
blending them by using the additive value function. It allows and provides a 
measure of consistency. AHP is also considered by some to be a better 
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method for developing a unidimensional value function by calculating the 
decision-maker’s preference on a ratio scale [Kamenetzky 1982]. The 
mathematical operations (multiplication, addition, etc.) applied to a ratio scale, 
unlike the interval scale, will produce a meaningful result [Saaty 1990]. 
Theoretical work on AHP is found in Harker and Vargas [1987], Harker [1987], 
and Saaty 1980, 1986]. A brief description of the method is offered in 
Appendix B. 

Schoemaker and Waid [1982] conducted an experimental comparison of 
five approaches for determining the weights of alternatives in a utility model. 
The experiment was run with 36 Wharton School sophomores taking an 
introductory decision science course. This group had been asked to evaluate 
multiple regression, direct trade-offs, and AHP methods in terms of perceived 
difficulty and trustworthiness. AHP was perceived to be the easiest and most 
trustworthy method to use. AHP is also very useful in building an analytical 
decision procedure for traditional capital budgeting. It allows evaluation of 
multiple decision criteria that can be tangible, intangible, quantitative, or 
qualitative [Stout et al. 1991]. 

AHP has been applied in a variety of planning and priority-setting situations 
(see e.g., Zahedi [1986], Golden et al. [1989], Golden and Wang [1989]), and 
it is also applicable to group decision-making [DeSanctis and Gallupe 1 985, 
DeSanctis 1987, Turban 1988]. 
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However, AHP is not without problems. Critics of this method claim that, 
in some situations, the ranking of alternatives determined by AHP may be 
changed by adding or deleting an alternative [Dyer 1990, Belton and Gear 
1983]. This phenomenon, which is known as rank reversal, is the most 
controversial issue with this method; however, the method is well defended by 
Saaty and other AHP loyalists [Saaty 1990, Harker and Vargas 1990]. 

Several explanations and solutions are proposed to eliminate or excuse this 
phenomenon. It has been claimed that rank reversal is a natural process and 
should be expected in real-life situations where the criteria are related to the 
alternatives under consideration [Saaty 1993, 1992, 1987; Harker 1990]. There 
are also numerous proposed revisions to the method that will eliminate this 
occurrence [Dyer 1990, Belton 1986, Sadrian and Kocaoglu 1986]. In this 
thesis, AHP is used to establish the relative weights of different outputs. 

5.2 Estimation of Output Weights W,, for TSD 

The weight of each output can be developed by using any of the 
methods mentioned previously. Each method has its own strengths and 
weaknesses, and each method is supported and criticized by various scientists. 
But all experts agree that the method should be suitable to decision-makers. 
This means that the method should be selected for its practical usefulness, 
ease of understanding and implementing, and validity. An introductory 
explanation to the decision-makers of the methods under consideration resulted 
in their selecting AHP for conducting an experiment. This method was selected 
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because it is easy to apply and comprehend. Also, the decision-makers were 
pleased with AH P’s capability of structuring the decision model. AHP has been 
applied in many governmental and institutional situations. The most successful 
applications have been in group decision-making where the group structures 
the problem in a hierarchy framework and pairwise comparisons are elicited 
from each level of the hierarchy. However, the number of comparisons can 
become overwhelming. The number of comparisons is n(n - 1)/2, where n is 
the number of outputs. There are 58 outputs X H that must be compared, 
resulting in over 1600 comparisons. 

To reduce the number of comparisons, TSD’s objective statement was 
developed. Then, a set of attributes or criteria that further define the objective 
of the directorate was generated. Attributes were compared with each other 
against the directorate objective. Next, divisions were compared on the basis 
of their contribution to the attributes that constitute the directorate’s objective. 

AHP was implemented by presenting two attributes at a time to the 
decision-makers and attempting to reach agreement among them on the ratio 
of weights between the two attributes. The outcome of this process is a set of 
weights for each division with respect to the directorate objective (this process 
is presented graphically later in this chapter). The outputs of each division are 
ranked by assigning points or by administering an AHP at the division level. 

With this multilevel assessment, the weight of an output is acquired by 
multiplying its conditional weight estimated within its own division by the weight 



73 


of the division itself. This procedure will reduce the number of comparisons 
significantly and present a practical task for decision-makers. 

A meeting was scheduled with the division chiefs and the director. The 
purpose of the meeting was to become familiar with AHP by implementing 
comparison techniques to develop a relative weight for each division. Before 
the meeting, a set of preliminary criteria was developed and E-mailed to the 
division chiefs to give some examples of criteria and to stimulate their thoughts 
for additional discussion. 

The first step after presenting the agenda was to offer a brief explanation 
of the method. The objective of the directorate was decided on, to be the best 
provider of service to the R&D organizations of the Center. Decision-makers 
determined that in order to be the best service provider, TSD must be the best 
in providing hardware (HWD), facilities (FAC), health and a safe environment 
(H/S/E) at the workplace, and energy (ENERGY). Hardware means tools, 
equipment and models that R&D needs to run the necessary tests. Facilities 
are defined as the research rigs, wind tunnels, and office complexes where 
research is performed. Health and a safe environment for employees is 
another important ingredient for a successful R&D organization. The last 
important element is uninterrupted energy sources to run the research. 

The problem is demonstrated in AHP format based on Saaty [1982] (see 
Table 5.1). The main objective occupies the top level, which is called the 
focus. The intermediate level comprises several elements that can best 
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advance the objective and are compared with one another against the focus. 
The lowest level of the structure comprises the divisions, which are compared 
on their contributions to the elements in the intermediate level. 

The six divisions were compared with respect to four attributes. To 
demonstrate the comparison procedure, I will explain the comparison matrix for 
hardware (HWD). According to Saaty [1982], first we draw a matrix with HWD 
in the upper left-hand comer. Then, the divisions are listed in the left column 
and on the top row (Table 5.2). Before questions are asked, the budget of 
each division should be known by all the decision-makers in order to help them 
make informed decisions on the comparison matrices. 

The 6-by-6 comparison matrix has 36 entries. How much does one division 
contribute to HWD relative to another division? The diagonal entries are 1 .00 
because there each division is compared with itself. From the division chiefs’ 
experience and preference, the values of the other 30 entries were provided 
according to the divisions’ contribution to HWD. 

In this type of matrix, the elements that appear in the left-hand column are 
always compared with the elements appearing in the top row. The value is 
given to the element in the column as it compares with the element in the top 
row. If the element in the left-hand column is less favorable, the value is a 
fraction. The reciprocal value is entered in the position where the element in 
the left-hand column appears in the top row and the element in the top row 
appears in the left-hand column. For example in Table 5.2, FPO contributes 
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one-ninth of the FSD contribution and one-eighth of TID contribution to HWD. 
This means that FSD contributes 9 times more than FPO and TID contributes 
8 times more that FPO, as recorded in the first row and first column of the 
matrix. 

Next, the matrix is normalized by dividing each entry by the total of its 
column. Finally, each row is averaged by adding the values in each row of the 
normalized matrix and dividing by the number of entries in each row. This 
operation yields the fraction of overall contribution of each division with respect 
to the HWD, which is called the priority vector (Table 5.3). In the mathematical 
context the comparison matrix is a single-ranked matrix because every row is 
a multiplier of the first row. 

All entries of this matrix are positive, and the matrix has reciprocity, where 
an entry c tj is equal to l/c^. According to Saaty [1977], a single-ranked 
reciprocal matrix with positive entries is consistent if and only if the only 
eigenvalue of the comparison matrix is equal to n, where n is the number 
of objects compared. A measure for consistency is established to be 

CR = (>w - n )/(n - 1) (5-2) 

In order to approximate X max for an inconsistent matrix, such as a 
comparison matrix, every column is multiplied by the relative priority of the 
division from the priority vector. For example, in Table 5.4 the FPO column is 
multiplied by the FPO element of the priority vector (0.03). Next, each element 
in the column “ROW TOTAL” is divided by its corresponding element of the 
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priority vector. If the matrix was 100 percent consistent, all the values in the 
new column (RT/PV) would have been n, which is 6 here. The average value 
of this new column (RT/PV) is the estimation of [Saaty 1982]. 

Now that A max is estimated, the value of CR is known. Saaty and Mariano 
[1979] found mean inconsistency for samples of 500 random filled matrices of 
each size from 2-by-2 to 10-by-10 matrices. The numerical judgments were 
taken, at random, from the scale 1/9, 1/8, 1/7, ..., 1/2, ..., 1, 2, ..., 9. Then by 
using a reciprocal matrix, the following average consistencies for different-order 
random matrices were determined: 

Size of matrix 12345 6 7 8 9 10 

Random consistency 0 0 .58 .9 1.12 1.24 1.32 1.41 1.45 1.49 

Cl is the ratio of CR over the appropriate random consistency and is 
recommended to be 0.1 or less for a consistent judgment. A decision based 
on inconsistent judgment will not be accurate and should be avoided. 

Following the same procedures, a priority vector for all other elements in 
the intermediate level (FAC, H/S/E, ENERGY) was developed. Next, a 
comparison matrix was developed where these elements were compared with 
each other against the objective of the directorate. The range of each attribute 
value was recognized by decision-makers as evidenced from the consensus for 
various scores in the comparison matrix. The final result of the process is 
presented in the final matrix. 
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The results of this exercise, which are given in Appendix B, raised some 
concerns. Decision-makers were troubled with the fact the OEP’s weight (0.1 5) 
was higher than FOD’s weight (0.12); yet most of TSD’s resources are 
appropriated to FOD. They believed that an organization supporting the R&D 
operation should have higher priority than the environmental program. Further 
investigation revealed that facilities and operation was not included in the 
previous attributes. In this context, facilities and operation is the element that 
captures the efforts of TSD in providing facilities and maintaining central 
systems, such as a steam plant and a compressed air system, to support R&D 
activities. 

Therefore, another AHP was conducted, like the previous one, with five 
attributes: hardware (HWD), facilities and operation (FAC/OPER), health and 
safe environment (H/S/E), energy (ENERGY), and R&D operation (R&D OPER) 
(See Table 5.1). The R&D operation accounts for TSD’s efforts in running the 
research rigs and R&D testing. The second AHP produced a divisional ranking 
(Table 5.20) that was reasonable and acceptable to the division chiefs. Tables 
5.5 to 5.20 present the results of the final ranking for the divisions. 

Table 5.1 Second Hierarchy for TSD Outputs 

FOCUS: TSD MISSION R&D TEST 

ATTRIBUTES: HWD FAC/OPER H/S/E ENERGY R&D OPER 


DIVISIONS: 


FPO OEP TID FOD FSD FED 
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Table 5.2 Initial Weight Ratio (Column/Row) for Hardware 


HWD 

FPO 

OEP 

TID 

FOD 

FSD 

FED 

FPO 

1.00 

0.33 

0.13 

0.33 

0.11 

0.33 

OEP 

3.00 

1.00 

0.20 

1.00 

0.13 

3.00 

HD 

8.00 

5.00 

1.00 

7.00 

0.20 

7.00 

FOD 

3.00 

1.00 

0.14 

1.00 

0.11 

1.00 

FSD 

9.00 

8.00 

5.00 

9.00 

1.00 

9.00 

FED 

3.00 

0.33 

0.14 

1.00 

an 

1.00 

TOTAL 

27.00 

15.67 

6.61 

19.33 

1.66 

21.33 


Cl = 0.08 


Table 5.3 Normalized Matrix With Eigenvector for Hardware 


HWD 

FPO 

OEP 

TID 

FOD 

FSD 

FED 

PRI 

VECTOR 

(PV) 

FPO 

0.04 

0.02 

0.02 

0.02 

0.07 

0.02 

0.03 

OEP 

0.11 

0.06 

0.03 

0.05 

0.08 

0.14 

0.08 

TID 

0.30 

0.32 

0.15 

0.36 

0.12 

0.33 

0.26 

FOD 

0.11 

0.06 

0.02 

0.05 

0.07 

0.05 

0.06 

FSD 

0.33 

0.51 

0.76 

0.47 

0.60 

0.42 

0.52 

FED 

0.11 

0.02 

0.02 

0.05 

0.07 

0.05 

0.05 

TOTAL 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 
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Table 5.4 Pairwise Comparison Matrix for Hardware 


HWD 

fpctpv 

OEP*PV 

TID*PV 

FOD*PV 

FSD*PV 

FED'PV 

ROW 

TOTAL 

RT/PV 

FPO 

0.03 

0.03 

0.03 

0.02 

0.06 

0.02 

0.18 

6.23 

OEP 

0.09 

0.08 

0.05 

0.06 

0.06 

0.16 

0.50 

6.40 

TID 

0.24 

0.39 

0.26 

0.42 

0.10 

0.37 

1.79 

6.81 

FOD 

0.09 

0.08 

0.04 

0.06 

0.06 

0.05 

0.38 

6.23 

FSD 

0.27 

0.63 

1.31 

0.54 

0.52 

0.48 

3.75 

7.28 

FED 

0.09 

0.03 

0.04 

0.06 

0.06 

0.05 

0.32 

6.07 


AVR.: 6.50 


Table 5.5 Initial Weight Ratio (Column/Row) for Facilities and Operations 


FAC/OPER 

FPO 

OEP 

TID 

FOD 

FSD 

FED 

FPO 

1.00 

7.00 

7.00 

0.33 

9.00 

1.00 

OEP 

0.14 

1.00 

3.00 

0.02 

5.00 

0.14 

TID 

0.14 

0.33 

1.00 

0.20 

3.00 

0.14 

FOD 

3.00 

5.00 

5.00 

1.00 

6.00 

1.00 

FSD 

0.11 

0.20 

0.33 

0.17 

1.00 

0.14 

FED 

1.00 

7.00 

7.00 

1.00 

7.00 

1.00 

TOT A 1 

5.40 

20.53 

23.33 

2.90 

31.00 

3.43 


TOTAL 


Cl = 0.10 
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Table 5.6 Normalized Matrix With Eigenvector for Facilities and Operations 


FAC/OPER 

FPO 

OEP 

TID 

FOD 

FSD 

FED 

PRI 

VECTOR 

FPO 

0.19 

0.34 

0.30 

0.11 

0.29 

0.29 

0.25 

OEP 

0.03 

0.05 

0.13 

0.07 

0.16 

0.04 

0.08 

TID 

0.03 

0.02 

0.04 

0.07 

0.10 

0.04 

0.05 

FOD 

0.56 

0.24 

0.21 

0.34 

0.19 

0.29 

0.31 

FSD 

0.02 

0.01 

0.01 

0.06 

0.03 

0.04 

0.03 

FED 

0.19 

0.34 

0.30 

0.34 

0.23 

0.29 

0.28 

TOTAL 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 


Table 5.7 Pairwise Comparison Matrix for Facilities and Operation 


FAC/OPER 

FPO*PV 

OEP*PV 

T!D*PV 

FOD*PV 

FSD*PV 

FED*PV 

ROW 

TOTAL 

RT/PV 

FPO 

0.25 

0.55 

0.34 

0.10 

0.26 

0.28 

1.80 

7.08 

OEP 

0.04 

0.08 

0.15 

0.06 

0.15 

0.04 

0.51 

6.44 

TID 

0.04 

0.03 

0.05 

0.06 

0.09 

0.04 

0.30 

6.17 

FOD 

0.76 

0.40 

0.24 

0.31 

0.18 

0.28 

2.17 

7.05 

FSD 

0.03 

0.02 

0.02 

0.05 

0.03 

0.04 

0.18 

6.17 

FED 

0.25 

0.55 

0.34 

0.31 

0.21 

0.28 

1.94 

6.91 


AVR.: 


6.64 


Table 5.8 Initial Weight Ratio (Column/Row) 
for Health and Safe Environment 


H/S/E 

FRO 

OEP 

TID 

FOD 

FSD 

FED 

FRO 

1.00 

0.13 

0.14 

0.14 

0.20 

0.14 

OEP 

8.00 

1.00 

3.00 

3.00 

6.00 

4.00 

TID 

7.00 

0.33 

1.00 

1.00 

3.00 

3.00 

FOD 

7.00 

0.33 

1.00 

1.00 

4.00 

2.00 

FSD 

5.00 

0.17 

0.33 

0.25 

1.00 

3.00 

FED 

7.00 

0.25 

0.33 

0.50 

0.33 

1.00 

TATAI 

35.00 

2.21 

5.81 

5.89 

14.53 

13.14 


TOTAL 

Cl = 0.10 


Table 5.9 Normalized Matrix With the Eigenvector 
for Health and Safe Environment 


H/S/E 

FPO 

OEP 

TID 

FOD 

FSD 

FED 

PRI 

VECTOR 

FPO 

0.03 

0.06 

0.02 

0.02 

0.01 

0.01 

0.03 

OEP 

0.23 

0.45 

0.52 

0.51 

0.41 

0.30 

0.40 

TID 

0.20 

0.15 

0.17 

0.17 

0.21 

0.23 

0.19 

FOD 

0.20 

0.15 

0.17 

0.17 

0.28 

0.15 

0.19 

FSD 

0.14 

0.08 

0.06 

0.04 

0.07 

0.23 

0.10 

FED 

0.20 

0.11 

0.06 

0.08 

0.02 

0.08 

0.09 

TOTAL 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 
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Table 5.10 Pairwise Comparison Matrix for Health and Safe Environment 


H/S/E 

FPO'PV 

OEP*PV 

TID*PV 

FOD*PV 

FSD*PV 

FED*PV 

ROW 

TOTAL 

RT/PV 

FPO 


0.05 

0.03 

0.03 

0.02 

0.01 

0.16 

6.21 

OEP 

0.21 

0.40 

0.56 

0.56 

0.62 

0.37 

2.72 

6.74 

TID 

0.19 

0.13 

0.19 

0.19 

0.31 

0.28 

1.28 

6.81 


0.19 

0.13 

0.19 

0.19 

0.41 

0.18 

1.29 

6.91 

FSD 

0.13 

0.07 

0.06 

0.05 

0.10 

0.28 

0.69 

6.72 

FED 

0.19 

0.10 

0.06 

0.09 

0.03 

0.09 

0.57 

6.15 


AVR.: 6.59 


Table 5.1 1 Initial Weight Ratio (Column/Row) for Energy 



FPO 

OEP 

TID 

FOD 

FSD 

FED 

FPO 

1.00 

3.00 

4.00 

0.17 

4.00 

1.00 

OEP 

0.33 

1.00 

4.00 

0.33 

4.00 

3.00 

TID 

0.25 

0.25 

1.00 

0.14 

3.00 

0.33 

FOD 

6.00 

3.00 

7.00 

1.00 

7.00 

3.00 

FSD 

0.25 

0.25 

0.33 

0.14 

1.00 

0.17 

FED 

1.00 

0.33 

3.00 

0.33 

6.00 

1.00 


TOTAL 8.83 7.83 19.33 2.12 25.00 8.50 


Cl = 0.1 1 
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Table 5.12 Normalized Matrix With Eigenvector for Energy 


ENERGY 

FPO 

OEP 

TID 

FOD 

FSD 

FED 

PRI 

VECTOR 

FPO 

0.11 

0.38 

0.21 

0.08 

0.16 

0.12 

0.18 

OEP 

0.04 

0.13 

0.21 

0.16 

0.16 

0.35 

0.17 

hd 

0.03 

0.03 

0.05 

0.07 

0.12 

0.04 

0.06 

FOD 

0.68 

0.38 

0.36 

0.47 

0.28 

0.35 

0.42 

FSD 

0.03 

0.03 

0.02 

0.07 

0.04 

0.02 

0.03 

FED 

0.11 

0.04 

0.16 

0.16 

0.24 

0.12 

0.14 

TOTAL 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 


Table 5.13 Pairwise Comparison Matrix for Energy 


ENERGY 

FPO*PV 

OEP*PV 

71D*PV 

FOD*PV 

FSD*PV 

FED'PV 

ROW 

TOTAL 

RT/PV 

FPO 

0.18 

0.52 

0.23 

0.07 

0.14 

0.14 

1.27 

7.18 

OEP 

0.06 

0.17 

0.23 

0.14 

0.14 

0.41 

1.15 

6.61 

TID 

0.04 

0.04 

0.06 

0.06 

0.10 

0.05 

0.35 

6.24 

FOD 

1.06 

0.52 

0.40 

0.42 

0.24 

0.41 

3.05 

7.23 

FSD 

0.04 

0.04 

0.02 

0.06 

0.03 

0.02 

0.22 

6.56 

FED 

0.18 

0.06 

0.17 

0.14 

0.20 

0.14 

0.89 

6.44 


AVR.: 


6.71 
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Table 5.14 Initial Weight Ratio (Column/Row) for R&D Operation 


R&D 

OPER 

FPO 

OEP 

T1D 

FOD 

FSD 

FED 

FPO 

1.00 

1.00 

0.11 

0.14 

0.33 

1.00 

OEP 

1.00 

1.00 

0.13 

0.14 

0.33 

0.33 

T1D 

9.00 

8.00 

1.00 

7.00 

7.00 

9.00 

FOD 

7.00 

7.00 

0.14 

1.00 

1.00 

5.00 

FSD 

3.00 

3.00 

0.14 

1.00 

1.00 

3.00 

FED 

1.00 

3.00 

0.11 

0.20 

0.33 

1.00 

TOTAL 

22.00 

23.00 

1.63 

9.49 

10.00 

19.33 


Cl = 0.08 


Table 5.15 Normalized Matrix With Eigenvector for R&D Operation 


R&D 

OPER 

FPO 

OEP 

T1D 

FOD 

FSD 

FED 

PRI 

VECTOR 

FPO 

0.05 

0.04 

0.07 

0.02 

0.03 

0.05 

0.04 

OEP 

0.05 

0.04 

0.08 

0.02 

0.03 

0.02 

0.04 

T1D 

0.41 

0.35 

0.61 

0.74 

0.70 

0.47 

0.55 

FOD 

0.32 

0.30 

0.09 

0.11 

0.10 

0.26 

0.20 

FSD 

0.14 

0.13 

0.09 

0.11 

0.10 

0.16 

0.12 

FED 

0.05 

0.13 

0.07 

0.02 

0.03 

0.05 

0.06 

TOTAL 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 
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Table 5.16 Pairwise Comparison Matrix for R&D Operation 

ROW 

FPO'PV OEP'PV TID'PV FOD*PV FSD*PV FED*PV TOTAL RT/PV 

0.04 0.06 0.27 6.25 

0.04 0.02 0.24 6.14 

0.83 0.53 3.97 7.27 

0.12 0.29 1.25 6.41 

0.12 0.18 0.81 6.81 

0.04 0.06 0.36 6.11 

AVR.: 6.50 


Table 5.17 Initial Weight Ratio (Column/Ratio) for R&D Test 


R&D TEST 

HWO 

FAC/OPER 

H/S/E 

ENERGY 

R&D 

OPER 

HWO 

1.00 

8.00 

5.00 

7.00 

0.50 

FAC/OPER 

0.13 

1.00 

1.00 

1.00 

0.33 

H/S/E 

0.20 

1.00 

1.00 

4.00 

0.14 

ENERGY 

0.14 

1.00 

0.25 

1.00 

1.00 

R&D OPER 

2.00 

3.00 

7.00 

1.00 

1.00 

TOTAL 

3.47 

14.00 

14.25 

14.00 

2.98 


Cl = 0.08 


Table 5.18 Normalized Matrix With Eigenvector for R&D Test 


R&D TEST 

HWO 

FAC/OPER 

H/S/E ENERGY 

R&D 

OPER 

PRI 

VECTOR 

HWO 

0.29 

0.57 

0.35 

0.50 

0.17 

0.38 

FAC/OPER 

0.04 

0.07 

0.07 

0.07 

0.11 

0.07 

H/S/E 

0.06 

0.07 

0.07 

0.29 

0.05 

0.11 

ENERGY 

0.04 

0.07 

0.02 

0.07 

0.34 

0.11 

R&D OPER 

0.58 

0.21 

0.49 

0.07 

0.34 

0.34 

TOTAL 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 



R&D 

OPER 



Table 5.19 Pairwise Comparison Matrix for R&D Test 
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On a request of OEP’s division chief, the same procedure was executed for 
OEP to develop the rank of outputs. The range of outputs was known before 
the exercise, and they are defined later in this chapter. AHP was applied at the 
division level by presenting two attributes at a time to the decision-makers of 
the division and attempting to reach agreement on the ratio of weights between 
the two attributes. This approach presented a few problems. The branch 
chiefs had difficulty deciding on the objective of the division. It was hard to 
develop consensus on the attributes of the objective, and voting on the outputs 
and attributes was inconsistent. 

The difficulties with the implementation of AHP at this level can be 
attributed to the limited knowledge of each branch chief of the other branches’ 
activities. Also, branch chiefs are too involved with the activities in their own 
branches to have an objective opinion on the other branches’ importance and 
worth to the division. The division chief, on the other hand, has a perfect 
bird’s-eye view of the entire division and is apprised of the directorate priority 
enabling him or her to evaluate the outputs more or less objectively. For these 
reasons, it was decided to involve only the division chief in ranking the outputs 
of the divisions. 

This was considered a very successful implementation of AHP where 
decision-makers were very comfortable and enthusiastic with the method. The 
steps of the technique were easily understood and accepted as a valid 
procedure for constructing the decision model. However, the validity of the 
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method has been questioned by some of the theorists in the decision analysis 
field [Dyer 1 990, Belton and Gear 1 983]. 

To produce valid weights from this experiment, two measures were taken: 
(1) the range of each output was displayed while decision-makers were 
comparing various outputs and (2) the rank of the divisions produced by AHP 
was examined by using the ‘swing weighting' method. For properly examining 
the validity of AHP, it is essential to present the rank of outputs from the swing- 
weighting method before calculating the final rank from the AHP method and 
after developing comparison matrices. 

In the swing-weighting method, the decision-maker is asked the question, 
If all divisions were at their worst contribution levels to the TSD objective and 
it is possible to move only one of the divisions to its best level of contribution, 
which division would be the most desirable to move to its best level? After 
elevating the most desirable one to the best possible outcome, the decision- 
maker is then asked what would be the second most desirable one to move, 
and so on [Fast and Looper 1988]. 

This procedure was administered with each division chief individually, and 
every one of them ranked the divisions in three general groups of high, 
medium, and low priority levels. The divisions and their weights created by 
implementing AHP are shown in parentheses here. The high-priority group 
consists of TID (0.31) and FSD (0.25), the medium-priority group includes FOD 
(0.18) and OEP (0.1 1), and the low-priority group includes FED (0.08) and FPO 



(0.07). The swing-weighting procedure provided a rank order of the divisions 
independent from AHP that concurs with the results of AHP. 

5.3 Upper and Lower Bounds of Outputs UXq, LXq 

In this section, the limits of outputs are presented, and the value of outputs 
for a typical year, which I call the base year, are offered. The base year for 
TSD is fiscal year 1992. The base year data are required to establish the 
resources used as a function of the level of output produced. 

Let X if be the amount of output j that is produced by division i using various 
resources. Let LIX^ and LX^ be the maximum and minimum limits of Xq, 
respectively. A typical minimum level of output is usually set by regulations. 
For example, reports on energy consumption at Lewis are filed quarterly with 
NASA Headquarters. Also, a system’s requirements may impose a minimum 
level of an output. For example, repairs on roads, roofs, or steam pipes are 
dictated by their age and condition and not by a management decision. UX^ 
is usually set by policies outside the organization’s controls, such as restriction 
on the size of government. 

Let BX (j be the amount of output that is produced in a particular year (the 
base year). These data are required to determine the resource use function 
of outputs RXjj. The following questions were presented to the appropriate 
supervisors who oversee the production of the output in TSD divisions: 

1 . What is the definition of your output? 

2. What is the measuring unit of your output? 
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3. How many units of output were produced in fiscal year 1992 (BXjj)? 

4. What is the maximum number of units that can be produced (UX^)? 

5. What is the minimum number of units that should be produced (LX^)? 
The answers to these questions and the weight for each output are presented 
in Tables 5.21 to 5.27. These weights are division chiefs’ judgments on the 
importance of each output. The weight of each suboutput, such as X22A in 
Table 5.22, is determined by dividing the output X22 by the number of 
suboutputs, which is four in this case. The Facilities Operations Division (FOD) 
outputs are divided into two tables (5.24 and 5.25): institutional support and 
R&D support. 
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Table 5.24 Facilities Operations Division 
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Table 5.26 Fabrication Support Dlvl8lon Outputs 
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Chapter 6 

Resource Use Functions 

In this chapter, the resource constraint set is formulated by establishing the 
resource use functions for every output from the base-year data. In 
Section 6.1, the basis for calculating the resource use functions for output 
and quality S ( is presented. In Sections 6.2 and 6.3, the methods of estimating 
the output and quality resource use functions, respectively, are described. In 
Section 6.4, the mathematical model for TSD in its final form is presented. In 
Section 6.5, it is proven that the local optimum is the global optimum of the 
model. 

6.1 Introduction 

The organization is assumed to have several divisions, and the divisions 
are considered to be black boxes where internal policies and the intricacy of 
the mechanisms that drive a division are not examined. Instead, the 
relationships between the inputs RX and RS and the output quantity X and 
quality S are defined for the base year. These relationships, which I call 
resource use functions, should be ideally estimated from empirical data. 

This empirical approach searches for historical data from several years and 
then draws a pattern between resources and production. If historical data on 
customer satisfaction or outputs are not available (as is true with most 
government organizations), engineering analysis might be used to estimate 
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resource use functions. In engineering analysis, the analyst follows the 
production process every step of the way and measures the resources, such 
as time, money, and people, required to produce various levels and quality of 
output. This process is repeated several times to validate the resource use 
function and is obviously time consuming for an organization with numerous 
products and not practical for this study. 

The only viable option was to use expert judgment to determine the 
parameters. The question presented to the experts was, In your judgment, 
what resources are used to produce one unit of output in the base year (fiscal 
year 1 992)? Once this question was answered, a data point on the chart of 
resource use versus production X (i was determined. The second point was 
assumed to be at zero production and zero cost. Similarly, two points can be 
identified on the resource-use-versus-quality S; chart. It is evident that 
numerous functions can be generated from each chart that include these two 
points and that different functions might produce substantially different results. 

The inputs of the divisions were collected by interviewing the decision- 
makers or by inspecting the current budget allocation. The resource are listed 
in Appendix D for the base year. The eight types of resources are all money 
and people, which are assumed to be not interchangeable. The following are 
various resources, their types, and their indices: 

K = 1 Civil servant (FTE) 

K = 2 Support service contractor (SSC) 
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K = 3 Institutional budget ($ ROS) 

K = 4 Program support budget ($ FS9) 

K = 5 Co-ops (COP) 

K = 6 Training budget ($ TRN) 

K = 7 Travel budget ($ TRV) 

K = 8 Equipment budget ($ EQP) 

To establish the cost of maintaining quality RSJS,), an assumption was 
made that the resources allocated to manage the directorate are the cost of 
quality. The remaining resources appropriated to the division were considered 
to be the cost of producing the divisions output RX ijk (X jj ). In the following 
sections, detailed procedures are presented to establish the production and 
quality resource use functions for TSD. 

6.2 TSD Production Resource Use Function RX^X,,) 

In general, three types of production resource use functions can be 
assumed: concave, convex, and linear (Figure 6.1). 

If the function is assumed to be exponential (a special form of convex), it 
is assumed that after a certain point the production cost increases slightly as 
production increases dramatically, as is sometimes characteristic of a mass 
production organization. This assumption will not be valid in an organization 
such as NASA, where products are one of a kind. The asymptotic (a special 
form of concave) assumption of the production resource use function is valid 
only if the organization has reached the point of diminishing returns. This 



101 


important principle of production theory states that adding labor and variable 
resources (money and people) to a fixed capital (heavy machinery and 
facilities) may increase return per unit of input initially when capital is 
underutilized. But once the fixed capital is used efficiently, additional variable 
input will decrease the rate of production to a point that adding resources will 
reduce output [Wachtel 1988]. Therefore, if this condition exists, a maximum 
limit of production should be imposed for the sake of efficiency. 


PRODUCTION vs. RESOURCES 

FOR ANNUAL CONSTRUCTION OF FACILITIES BUDGET REQUEST 



Figure 6.1 Production Resource Use Function. 

After interviewing the decision-makers at TSD, it was determined that TSD’s 
production has not reached the point of diminishing returns. Actually, with 
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current fiscal constraints and budget cuts, we can assume that it will be difficult 
to find any government organization which has reached that point. Therefore, 
an asymptotic resource use function would not be a good assumption for this 
problem. A simple and plausible assumption for the production resource use 
function is that it is linear or proportional. With these assumptions 

RXjj(Xjj) = RXj jk Xjj (6-1) 

where RX^ is the amount of resource k that is needed to produce X^. To 
estimate RX ijk , the amount of resources that are used in the base year BRX jjk 
is divided by the number of outputs produced BXjj in that year. 

RX ijk = BRX jjl /BX i (6-2) 

For example, as shown in Table 6.1, the output XII has a base-year value 
(BX in ) of sixteen $1 million dollars in projects that were planned by two FTEs 
(BRX in ) in fiscal year 1992. The RX t11 , which stands for the resource use 
function of input type 1 used to produce the output 1 from division 1 , can be 
calculated by using equation (6-2): 

RX ni = 2/16 (FTE/Million Dollar Projects) 

In Tables 6.1 to 6.7, the resources exclusively dedicated to production and the 
outputs produced from these resources are specified for each division for fiscal 
year 1992. The Facilities Operations Division (FOD) activities are divided into 
two tables (6.4 and 6.5): institutional support and R&D support. 
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Table 6.6 FY92 Resources Allocated to 7400 (Fabrication Support Division) for Production Only 
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6.3 TSD Quality Resource Use Function RSJS,) 

The same dilemma in developing production resource use functions also 
exists with the quality resource use function. Only two data points can be 
identified. As explained before, three types of quality resource use functions 
can be assumed: concave, convex, and linear. If the quality resource use 
function is assumed to be exponential (a special form of convex), it is assumed 
that after a certain point the quality cost remains unchanged as quality 
increases. This assumption is obviously unreasonable. 

If the quality resource use function is assumed to be linear, the cost of 
increasing present quality per unit of quality is the same whether the present 
quality is near its lowest or highest value. This assumption is not valid 
because, intuitively, the marginal cost of increasing quality should be less at the 
lower end of the quality scale than at the higher end. 

Therefore, the quality cost function can be reasonably assumed to be 
asymptotic (a special form of concave), meaning that the cost of quality 
increases rapidly as the quality approaches its maximum value (Figure 6.2). 
The relationship between resources and quality is assumed to be asymptotic 
with the following form: 


c _ c m RS,^,) wy*> 

S, — S max (1 — m ik ) + m ik 


(6-3) 


where S,^ is the maximum score (here, 5) that can be given in a 
questionnaire. RS ik (Sj) is the amount of resource of type k, which could be 
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money, people, or equipment, that is required to maintain quality at the S s level. 
The value of Sj at zero amount of resource is 1. 


QUALITY SCORE vs. RESOURCES 
FOR FACILITIES PLANNING OFFICE C^PCO 



FTE DEDICATED TO QUALITY 

Figure 6.2 Quality Resource Use Function. 


Equation (6-3) suggests that the closer the quality to its maximum level, the 
higher is the cost of maintaining it. Rearranging equation (6-3) gives 

RSJSJ = ln[(S max - S,)/^ - 1 )]/ln(m ik ) (6-4) 

The parameter m ik (<1 ) is calculated from quality data for the base year BSj 
and the resources devoted to maintaining quality during the base year 
BRSj k (BSj). BSj was estimated by averaging the general satisfaction score that 
each division received from the customers throughout the questionnaire. This 
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procedure indicates that the opinions of all customers are equally important for 
TSD. 

The value of BRS ik (BS;) was determined by estimating the amount of 
resources committed to improving the division’s customer satisfaction. For 
example, it was assumed that division chiefs, secretaries, branch chiefs, and 
people in any other administrative positions directly affect customer satisfaction 
by planning, reviewing, and assigning resources to various products of the 
organization. This assumption is valid because the managers in TSD are 
mainly responsible for providing the best service possible to the other 
directorates at the Center. 

Another assumption was that funding for training is also dedicated to 
improving customer satisfaction. This assumption is reasonable because the 
training budget is mostly associated with administration, as shown in Table 6.8. 

The travel budget was also credited to quality improvement because it is 
closely related to the training budget. Table 6.9 displays some costs of quality 


for the directorate. 
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Table 6.8 FY92 Training Instances by Percentage Within TSD 














Table 6.9 FY92 Resources Allocated to the Quality of Output In Various Divisions 
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6.4 TSD Quality Resource Use Function Parameter m* 

Now that resources and level of quality had been determined, the 
parameter m jk could be calculated by using equation (6-4) and the base-year 
resources used. On the asymptotic curve, equation (6-4), two points were 
identified. The first point was at quality level 1 and zero cost. The second 
point was generated by estimating resources and quality for the base year. 

For example, consider the division office for FSD (7400). The m 51 
(division 5 and resource 1) can be calculated by inputting the base-year values 
of the parameters BRS 51 (S 5 ) = 7, BS 5 = 4.28, and = 5 in equation (6-4), 
resulting in 

7 = [ln(5 - 4.28)/(5 - l^/lnfm*) (6-5) 

m 51 = e™ 0 72/4)1/7 = 0.783 (6-6) 

With the calculated value of m 51 , a general formulation for equation (6-4), the 
resource use function of FSD is estimated from the quality of the FSD output: 
RS 51 (S 5 ) = [ln(5 - S 5 )/(5 - 1 )]/ln(0.783) (6-7) 

Although these are not nearly enough points to construct a function that one 
can be fully confident in, it provides the best possible answer at this time. As 
more data become available, a more accurate resource use function can be 
estimated. With the above assumptions, a more specific version of the model 
is presented in the next section. 
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6.5 TSD Mathematical Model 

Based on the assumptions outlined earlier in Chapters 2 and 6, the TSD 
mathematical model in its final form is 

Max Z = X,S, ^WjX/OJX,) (6-8) 

subject to 

ZlnKS^-SMS^-ljyinfmJ 


XZi RX*X, 

Vk 

(6-9) 

, S S, £ 

Vi 

(6-10) 

S X, s ux, 

Vij 

(6-11) 


All variables are nonnegative. 

The decision variables are 

Sj General opinion of customers on the quality of division i outputs on a 
1 to 5 scale 

Xjj Units of output j produced by division i 
The parameters are 

B k Total resource of type k available to the organization 
LXjj Lower bound of 

m ik Quality cost parameter for division i and resource k 
RX jjk Units of resource k required to produce one unit of X js 
S max Upper bound of S ( 
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S,,*, Lower bound of Si 

UXjj Upper bound of 

Wjj Weight of output j of division i 

This problem has a convex feasible region but a nonconcave objective 
function. These conditions can make the problem nonconcave, where the local 
optima are not necessarily global optima. A problem is concave if the feasible 
region is convex and the objective function is strongly quasi-concave 
[Chankong 1989]. Global optimality can be proven for a two-dimensional 
version of the TSD problem where only the variables S 1 and X t1 are 
considered. However, the general problem, with n variables, does not 
necessarily produce a global optimum because adding quasi-concave functions 
will not necessarily result in a quasi-concave function. 


The following example is constructed to demonstrate that in the general 
TSD problem, local optima are not necessarily global optima and therefore the 
problem is nonconvex. 



Max Z = O^X, + Spc, 

(6-12) 

subject to 



N(Sm*, - S,V(S^ 

- 1)]/ln(m) + X, 



+ InKS^ - SJ/fS™, - 1))/ln(m) + X, < B, 

(6-13) 


Smin ^ S 1f S 2 < S max 

(6-14) 


0 < x 1f x 2 

(6-15) 
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where 



B, = 30 
m = 0.5679 


Two feasible solutions to this model are examined where one has a better 
value of the objective than the other. Then, the feasible point with the lower 
value of the objective is proven to be a local optimum, proving that the problem 
is nonconvex. 

The first solution is provided by fixing X 2 and S 2 values at zero and 
calculating the optimum value of S, and X 1 by using equations (6-1 2) through 
(6-15). The problem becomes 

Max Z = 0.9S 1 X 1 (6-16) 

subject to 

ln[(5 - SM5 - 1 )]/ln(0.5679) + X, 

+ ln[(5)/(5 - 1 )]/ln(0.5679) < 30 (6-1 7) 

1<S,<5 (6-18) 

X,>0 (6-19) 

The first solution resulted in S, = 4.68091, X t = 25.92543, S 2 = 0, X 2 = 0, and 
Z = 1 09.21 5. The second solution is determined by setting and X, equal to 
zero and solving equations (6-12) through (6-15). The second solution is 
S t = 0, X, = 0, S 2 = 4.68091 , X 2 = 25.92543, and Z = 121 .35. These solutions 
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are both feasible, but the second solution is clearly better because of its higher 
objective function value (121.35 versus 109.215). If it can be shown that the 
first solution (Z = 1 09.21 5) is a local optimum, the problem has a local optimum 
that is not globally optimal, and hence the problem is nonconvex. 

From the first solution, which is a feasible point, a search for a feasible 
direction to improve the objective function value is conducted. By following the 
method of Zoutendijk [1960] (page 418 of Bazaraa et al. [1993]) for nonlinear 
inequality constraint problems, the feasible direction is determined for a general 
problem: Minimize f(x) subject to gi(x) < 0 for i = 1,...,m. Let x be a feasible 

solution to the general problem, and let I = {i:gj(x) = 0}, where I is the index of 

binding constraints. The following direction-finding problem is defined: 

Minimize p (6-20) 

subject to 

Vf(x)*d - p < 0 (6-21) 

Vg^d - p < 0 for i e I (6-22) 

-1 < dj < 1 for j = 1 n (6-23) 

where {d,, d 2 , d 3 , d 4 } are decision variables. 

Applying this method to equations (6-12) through (6-15) for solution 1 


results in 


Minimize p 
{d-|, d 2 , d 3 , dj 


(6-24) 
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subject to 


-0.9S 1 d 1 - 0.9X 1 d 2 - S 2 d 3 - X 2 d 4 - p < 0 

(6-25) 

d2/[ln(0.5679)(5 - S,)] + d 3 - d 4 /[ln(0.5679) 


x (5 - S 2 )] - p < 0 

(6-26) 

~d 3 - p < 0 

(6-27) 

-d 4 - p < 0 

(6-28) 


-1 < dj < 1 forj = 1,.„,4 

Equations (6-18) and (6-19) are not binding for S, and X„ respectively, and 
only binding for S 2 and X 2 , resulting only in equations (6-27) and (6-28). 
Solving this problem for solution 1 (S, = 4.68091, X, = 25.92543, S 2 = 0, X 2 
= 0) resulted in no feasible direction for improving the objective function value, 
so that the feasible point is locally optimum. On the other hand, it is clear that 
this local optimum is not globally optimal because feasible solution 2 has an 
objective value of 121.35 (S, = 0, X, = 0, S 2 = 4.68091, X 2 = 25.92543, 
Z = 121 .35). Therefore, solution 1 , the local optimum, is not a global optimum 
and our problem, equations (6-8) through (6-12), is nonconvex. We can thus 
conclude that the solution found by GAMS is not necessarily a global optimum, 
since GAMS uses local gradient information to search for the optimal solution. 

Determining the global optimum for a nonconvex problem is a classical 
problem that is well researched. The methods that have been developed can 
be divided into two categories: deterministic methods and stochastic methods 
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[Dixon and Szego 1978]. Deterministic methods, which include trajectory 
methods [Brian 1972], the deflection method [Goldstein and Price 1971], and 
interval arithmetic methods [Hansen 1980], do not have any random or 
stochastic features. Stochastic methods, generally, sample the objective 
function at randomly selected points in the feasible region. These methods 
normally combine the random sampling with a phase where local minimization 
algorithms are performed from some of the sample points. Stochastic methods 
provide an attractive choice from the theoretical and computational points of 
view. A paper by Byrd et al. [1990] suggests that stochastic methods can be 
used to solve the global optimization problem while exploiting parallel 
algorithms. In particular, the problem can be decomposed into several 
independent problems and solved concurrently. 

In the first step of the Byrd et al. [1990] method, the feasible region R is 
divided into p equal-size subregions. The second step consists of a three- 
phase iteration. In the first phase of each iteration, the sampling phase, each 
processor generates 1/p of the sample points (where p is the number of 
processors) and evaluates the objective function at each point. The starting 
points, in the second phase, are selected by each processor from its own 
subsample space. In the third phase, local minimization is performed from all 
starting points. Each starting point is assigned to a processor, which performs 
a minimization from that point. Another starting point is assigned to the 
processor as soon as it terminates its current minimization. This procedure 
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continues until local searches from all starting points are completed. Then, in 
the third and final step, if the stopping rule is satisfied, the lowest local 
minimum is regarded as the global minimum. If the stopping rule is not 
satisfied, return to the first step. 


PART THREE 


RESULTS AND CONCLUSIONS 
Chapter 7 
Results 

In this chapter, the outcome of the model and several sensitivity analyses 
are presented. In Section 7.1, the solution procedure is outlined, and the 
methods and software used to solve this model are presented. In Section 7.2, 
the optimum and present resource allocations are presented in graphical form 
for each resource category (Figures 7.1 to 7.8) and are compared in 
Section 7.3. In Section 7.4, the impact of equal-versus-assigned weights on 
the results is examined, and in Section 7.5, the effect of budget size on the 
outcome is analyzed. In Section 7.6, the effect of two different quality resource 
use functions on the results is presented. In Section 7.7, the relative values 
of the various resources are presented. In Section 7.8, the results of the 
questionnaire are presented and the level of customer satisfaction is described. 
7.1 Solution Procedure 

The problem presented in Chapter 6, equations (6-8) to (6-11), is nonlinear 
with inequality constraints. The objective function is quadratic. The constraint 
set is also nonlinear because of the quality resource use function. As 
demonstrated in Section 6.5 the problem is convex, so any local optimum is 
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also a global optimum. Such problems can often be solved by using software 
such as GINO or GAMS. GAMS was used to solve the TSD model. 

GAMS is a compiler that can create large-scale optimization models. 
Nonlinear models are solved by employing a Fortran-based system called 
GAMS/MINOS. GAMS/MINOS uses a projected Lagrangian algorithm to solve 
these kinds of problems. A sequence of linearly constrained subproblems must 
be created and solved with the aim of converting the nonlinear constraints to 
linear ones. After defining a linearly constrained problem, GAMS/MINOS 
employs the reduced-gradient algorithm to minimize the objective function and 
find the local optimum. 

Nonsmooth and discontinuous functions should be avoided for this 
nonlinear solver. Also, integer restrictions cannot be imposed directly. 
GAMS/MINOS is designed to find local optima. The functions must be smooth 
(first derivatives exist and are continuous), but they need not be separable. 
See Murtagh and Saunders [1982] for additional information. The TSD model 
meets the above conditions and the model is optimized by using GAMS 
software. 

In implementing the TSD model, some of the outputs, such as garbage 
removal, security, and fire protection services, that did not have any range (i.e., 
LX = UX) were not considered, and their associated budget was removed from 


the model. 
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7.2 Optimum Resource Allocation Results 

The optimum resource allocation is presented in the form of several graphs. 
There is a chart for each type of resource (FTE, SSC, etc.) that presents the 
fiscal year 1992 level of resource committed to each division as well as the 
optimum level. These charts are presented in Figures 7.1 to 7.8, and are 
discussed below. 
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Figure 7.1 Full-Time Equivalents Required for Each Division. 
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Figure 7.2 Support Service Contractors Required for Each Division. 
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Figure 7.3 Fund Source 9 Money Required for Each Division. 
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Figure 7.4 Research Operation Support Required for Each Division. 
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Figure 7.5 Training Funds Required for Each Division. 
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Figure 7.6 Travel Funds Required for Each Division. 
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Figure 7.7 Equipment Funds Required for Each Division. 
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Figure 7.8 Co-Ops Required for Each Division. 

7.3 Optimum Versus Present Resource Allocation 

In this section, the optimum and present resource allocations are compared 
in each resource category. These comparisons are important because they 
provide insight into the shifts of resources among divisions. In Sections 7.3.1 
through 7.3.6, explanations are offered for the change in resources, the value 
of objective functions, and the quality and quantity of the outputs of various 
divisions. The results of the model directly depend on the inputs of the model: 
resource use functions, weight of outputs, definition of outputs, and methods 
of measuring outputs. The accuracy of the outcome is directly correlated with 
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the accuracy of the inputs of the model. The purpose of subsequent sections 
is merely to explain the results in order to illustrate the capability of the model. 

7.3.1 Analysis of FTE Allocation 

As illustrated in Figure 7.1 , the FPO and OEP divisions received more FTEs 
in the optimization; these FTE levels can be credited to the low cost of 
operation in these two divisions. Also, their customer satisfaction scores are 
relatively high, supporting the results of the model, which allocated more 
resources to these divisions in order to increase the number of satisfied 
customers. This result is demonstrated later in this section by an analysis of 
the constraint set and objective function coefficients of the model. 

Other divisions’ FTE levels, with the exception of FOD, did not differ 
significantly. The decrease in FOD’s FTE allocation can be attributed to this 
division’s having the most resources in TSD and not scoring very high in 
customer satisfaction. This decrease was to be expected, given that the model 
reduced the resources of FOD to compensate for the increase in other 
divisions. The same conclusion can be drawn for the SSC, ROS, CoOp, and 
EQP categories of resources. Only three divisions (OEP, TID, and FSD) 
needed CoOp resources. The above conclusions can be justified as follows. 

Mathematically, a partial derivative value of the objective function with 
respect to resources use of an output 0Z/dRX ijk (X jj ) is an indication of the 
resource cost of improving the objective. For instance, if this value is small for 
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Xjj and resource k relative to X iT for the same resource, more units of resource 
k will likely give more "bang for the buck" if devoted to X, r This partial 
derivative can be calculated as 

az/aRx^j) = (az/ax, j )[ax i /aRx jj (x s )] (7-1 ) 

Now, from equations (6-1) and (2-9), 

RX,(X,j) = RX^X, (7-2) 

Z = E.S, X,WJX/(UX,)] (7-3) 

Then, calculating the appropriate partial derivatives from equations (7-2) 

and (7-3) and substituting in equation (7-1) yield 

aZ/aRX^Xy) = (S j W i /UXy)(1/RXy) (7-4) 

For illustration purposes, consider the FTE (k = 1) allocation for an output 

from FPO, say XI 2, and another output from FOD, say X411 (Table 7.1). 


Table 7.1 Partial Derivative of Objective Function Value 
With Respect to Production Cost 


OUTPUT 

Si 

W/UX* 

RX* 


X12 

3.71 

0.001015 

0.0192 

0.196 

X41 1 

3.63 

0,000000288 

0.00007519 

0.01388 


The aZ/aRXyfXy) value in Table 7.1 indicates that if one additional FTE is 
allocated to XI 2, the objective function value will increase by 0.196, which is 
14 times more than the increase in the objective function value (0.01388) if the 
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same FTE is allocated to the output X411. Therefore, it is reasonable to 
expect that XI 2 will receive more FTEs than X411. This conclusion is shown 
in Figure 7.1, where FTE is reduced in FOD (X411) while increased in FPO 
(XI 2). 

7.3.2 Analysis of FS9, TRN, and TRV Allocations 

The optimum distribution of fund source 9 (FS9), k = 4, did not change from 
the current distribution, meaning that TSD presently uses FS9 in the best way 
according to the assumptions of the model. This conclusion was unexpected, 
but it may be attributed to the fact that TID, the division with the highest weight, 
does not require FS9 funding. The FS9 is provided by taxing the customers, 
who are the R&D directorates in the Center. This fund, in some cases, is 
monitored by customers, who give additional scrutiny in disbursing this 
resource. This result is interesting because if we believe that FS9 has an 
optimum allocation because customers have more control of this resource, we 
are supporting the market economy at Lewis. By "market economy," we mean 
that all the directorates are charged for the services that they receive from 
other directorates and there are no centralized budgets that fund special 
categories of services. An example of market economy is that every 
directorate at Lewis pays its own electric bill instead of TSD paying the entire 


electric bill for the Center. 
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The remaining resource categories, training (TRN) and travel (TRV), were 
increased for divisions with a high customer satisfaction score and decreased 
for the divisions with a low customer satisfaction score as shown in Tables 7.2 
and 7.3. This result was expected because the TRN and TRV budgets 
contribute only to maintaining customer satisfaction (see Tables 6.1 0 and 6.11). 


Table 7.2 Training Budget Versus Customer Satisfaction Score 



FPO 

OEP 

T1D 

FOD 

FSD 

FED 

PERCENT INCREASE 

127.5 

49 

17.1 

-30.7 

26.6 

-58.5 

QUALITY SCORE 
FROM SOLUTION 

4.696 

4.487 

4.405 

3.096 

4.596 

2.177 


Table 7.3 Travel Budget Versus Customer Satisfaction Score 



FPO 

OEP 

T1D 

FOD 

FSD 

FED 

PERCENT INCREASE 

127.4 

49 

17.5 

-30.7 

33.9 

-58.5 

QUALITY SCORE 
FROM SOLUTION 

4.969 

4.487 

4.405 

3.096 

4.596 

2.177 


7.3.3 Overall Customer Satisfaction 

Overall customer satisfaction Z, equation (2-9), is presented in Table 7.4 
for the entire directorate and its constituent divisions for the current and optimal 
resource allocations. This table represents the increase or decrease of 
objective function value for the directorate and divisions as a result of optimum 


resource allocation. 








































134 


Table 7.4 Objective Function Value tor Directorate and Divisions 


Objective Function 
Value 

^Direct 

^FPO 

^OER 


^FOO 

^FSO 

^FED 

Present 

2.94 

0.134 

0.206 

1.269 

0.382 

0.748 

0.196 

Optimum 

3.53 

0.326 

0.451 

1.294 

0.332 

0.889 

0.197 


The optimum resource allocation increased the directorate overall customer 
satisfaction by 20.1 percent (from 2.9 to 3.5). The overall customer satisfaction 
increased in all the divisions, except FOD, which shows a slight decrease. A 
shift in overall objective value function is due to a shift in quality and quantity 
of the division outputs. The next two sections explain the shift in quality and 
quantity of output in various divisions. 

7.3.4 Shift in Output Level by Division 

The shift in output level can be attributed to the resource use function RX^ 
and the weight of outputs in FOD. For more specifics, refer to Appendix C, 
where the result of a run is presented in four columns. Each decision variable 
is assigned four values, lower (lower bound of variable), level (the optimum 
value), upper (upper bound of variable), and marginal. 

The values in the marginal column are the amount of change in the 
objective function value, which we try to maximize, for one unit increase of 
variable. In other words, the marginal value of a variable is the partial 
derivative of the objective with respect to the variable. For instance, the 
marginal value for the variable XII (range 16 to 20) is 0.005, meaning that if 
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the variable XII could increase by one unit, the objective function value 
(customer satisfaction) will increase by 0.005, or 0.1 1 percent (0.005/3.53). 

The marginal values of the variables in FOD (X44A to X414) are mostly 
negative (because they are at their lower bounds) or very small positive 
numbers "EPS" (because they are between bounds). Therefore, increasing 
these variables (level of output) will not increase the objective function value 
(customer satisfaction) significantly and, in some instances, will decrease the 
objective function value. 

The question that one can ask is, How is it possible that increasing 
production can reduce customer satisfaction? The answer is that increasing 
production will result in increasing the resources to that output, which are 
calculated by the resource use function RX^. Because the resources of the 
directorate B k are limited, an increase of resource to an output with less 
contribution to the objective function value will reduce the resources to 
available outputs with a higher contribution. The outcome is an overall 
reduction in the objective function value. Therefore, the resource allocation is 
directly related to the marginal value of variables, and marginal values are 
determined from the cost of operation RX jjk and the weight of the output as 


illustrated in Table 7.1. 
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7.3.5 Shift in Quality of Output by Division 

This shift is due to the cost of customer satisfaction RS ik and the weight of 
output Wjj. As mentioned earlier (Section 6.3), the cost of maintaining the 
quality of a division is simplistically assumed to be the cost of managing the 
division. All else being equal, the model increases the division quality that has 
the lowest cost margin of quality improvement to gain the best results for 
resources used. 

As illustrated in Tables 6.9 and 6.10, FOD and FED had a relatively high 
cost of maintaining quality and a low quality score in the base year. For these 
reasons, the model allocated less resources to FOD and FED. 

I analyzed this last point mathematically as follows: a partial derivative 
value of the objective function with respect to resources use of an output 
9Z/9RS ik (Sj) has a positive correlation with the amount of resources allocated 
to the outputs: 

az/aRSjs,) = (az/as i )[as/aRs ik (s i )] (7-4) 

If equations (6-3), (6-4), and (2-9) are rewritten, 

s, = (1 - + mJW (7-5) 

RS*(S,) = InKS^ - S,)/(S„„ - 1 )]fln(mj (7-6) 

Z = I, ^ E,W,IX^(UX,)] (7-7) 

Then, from equation (7-5), 


dS/3RS„(S,) = 


(7-8) 
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Then, substituting RS jk (S i ) from equation (7-6) gives 

aS/3RS k (S,) = -4ln(m jk )((S max - S,)/(S„„ - 1)] (7-9) 

and from equation (7-7) 

azras, = w,[>v<ux,)] (7-1 o) 

Therefore, equation (7-4) becomes 

aZ/3RS ik (S i ) = {-4ln(m ik )[(S max - - 1 )]}{W,PV(UX,)D (7-11) 

In equation (7-11), the parameters S, and Xj f are the only variables, and all 
other parameters have a constant value. For illustration purposes, consider the 
FTE (k = 1) allocation for an output from FPO, say XI 2, and another output 
from FOD, say X411 (Table 7.5) 


Table 7.5 Partial Derivative of Objective Function 
Value With Respect to Quality Cost 


OUTPUT 

s, 

V^X/UX* 


az/aRS k (Si) 

X12 

3.71 

0.0162 

0.5679 

0.012 

X41 1 

3.63 

0.0306 

0.9596 

0.002 


The az/aRSJSj) value in Table 7.5 indicates that if one additional FTE is 
allocated to XI 2 for improving the quality, the objective function value will 
increase by 0.012, which is six times more than the increase in the objective 
function value (0.002) if the same FTE is allocated to the output X41 1 . 
Therefore, it is reasonable to expect that XI 2 will receive more FTEs than 
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X41 1 . This conclusion is exhibited in Figure 7.1, where FTE is reduced in FOD 
while increased in FPO. 

7.3.6 Shift in Resource Allocation 

The shifts in production levels and quality as explained previously require 
shifts in resource allocation among divisions. As explained in Section 7.3.2, the 
effect of each variable on the objective function value is the marginal value of 
the variable. Because the model is designed to maximize the objective function 
value, it is reasonable that variables with high positive marginal values will rise 
further to increase the objective function value. 

Because there is a direct relation between resources and the value of 
variables (linear for X Sj and asymptotic for Sj), an increase in the value of a 
variable is directly correlated with an increase in the allocated resources to the 
variable. This is the reason behind shifting resources from a variable with a 
lower marginal value to one with a higher marginal value. 

For example, consider the shift in the number of FTEs between FPO and 
FOD. Referring to Appendix C, the marginal values of the FPO outputs (X1 1 
to XI 6) are large positive numbers, whereas the marginal values of the FOD 
outputs are mostly negative or small positive numbers. Therefore, the model 
will be inclined to allocate enough resources for FPO to produce the maximum 
number of outputs in order to increase the objective function value. On the 
other hand, the resources to the FOD output, with the negative marginal value, 
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will be minimized in order to prevent the decline of objective function value. 
Indeed, this is so; FPO’s outputs are at their upper bounds and FOD’s are 
often at their lower bound. Hence, there will be fewer FTEs for FOD and more 
for FPO. 

7.4 Impact of Equal Versus Assigned Weights 

To examine the sensitivity of the results to the weights W^, I executed the 
model with equal importance on divisions to examine the impact of the weights 

on the resource allocation. The weight of each division ^LjWjj was assumed to 

be 1/6, or 16.7 percent. Then, this weight was distributed among the outputs 
of the division consistent with the internal priority of the division, which was 
decided by the division chief. The FTE allocation of this run is compared with 
the run for the nonequal-weight case in Figure 7.9. 

As demonstrated in Figure 7.9, the resource allocation model is sensitive 
to the weights of the divisions. I expected that outputs would increase as the 
weights of outputs increased and decrease as the weights of outputs 
decreased. The outputs of divisions changed as I expected, with the exception 
of FPO. The outputs of FPO are at their maximum levels with the assigned 
weight (0.07); and although the weight of the division increased to 0.167, the 
outputs remained constant because they were already at their upper bounds. 
These results point out the need for weights to be chosen with care. 
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Figure 7.9 Full-Time Equivalents Required With Assigned Versus Equal 
Weights for Each Division. (Note: Numbers above bars are division weights.) 


7.5 Impact of Budget Size on Results 

The budget B k was reduced by 10, 20, and 30 percent across all the 
resources k to observe the impact of budget reduction on the resource 
allocation. Also, the model was executed with a 10-percent budget increase. 
The result of this sensitivity analysis is presented in Figure 7.10. The figure 
does not include the 30-percent budget cut because the problem then becomes 
infeasible. Therefore, with the current structure of the organization and present 
constraints, the resource allocation cannot be optimized when the budget is cut 
by 30 percent. 
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Rgure 7.10 Full-Time Equivalents Required With Various Budget Levels. 

If the budget is cut by 10 percent, the FOD allocation will be cut the most 
because that division has the most resources in the directorate and not very 
high customer satisfaction rank among all other divisions. The objective value 
function decreases by 10 percent from the optimum value at the present 
budget level. Again, this determination strictly depends on the metrics that 
were developed for quantity and quality of outputs and the values assigned to 
them. 

If the budget is cut by 20 percent, the resource allocation follows the same 
pattern as the 10-percent cut but with more reduction. The optimum overall 
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customer satisfaction will drop to Z = 2.8 from Z = 3.5, which is the optimum 
overall customer satisfaction with the current budget level. Also, the overall 
customer satisfaction with a 20-percent cut will be 5 percent below the present 
level of customer satisfaction without optimization (Z = 2.93, Table 7.4). 

On the other hand, if the current budget is increased by 10 percent across 
the resources, all divisions receive near or above their current levels of 
resources, with the exception of FSD. The FTE allocation to FSD decreases 
in this scheme. Examining the output levels for each product of FSD revealed 
that the X56 output level is reduced from its upper bound to its lower bound. 
This change impacts the number of FTEs required for the output and reduces 
it to a level that supports the new level of X56, which is 26 percent above the 
current nonoptimum value and 6 percent above the current optimum value. 
Thus, the point of diminishing return has been reached because the objective 
function’s optimum value rose only 6 percent with a 10-percent increase in 
resources. This behavior was expected for the model because since there is 
a maximum limit for customer satisfaction regardless of the amount of input to 
the process owing to upper bounds on X s and S,. 

7.6 Impact of Quality Resource Use Function Formulation 

As an example of a sensitivity analysis with respect to the constraint set, 
two quality resource use functions RS were developed. One function was 
created with the assumption that with zero resources the quality score will be 
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zero. The other was created with the assumption that with zero resources the 
quality score will instead be 1 . The two resource use functions are similar 
especially for Sj > 3.5, which is where most divisions are. Figure 7.11 
illustrates two typical functions for FTE allocation to OEP. 
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Figure 7.1 1 Quality Resource Use Function for OEP. 


The objective function value for the formulation with the alternative function 
(zero resource implies a quality score of 1 ) equaled 3.529, about 0.5 percent 
higher than the one with zero resource that implies zero quality score (3.5075). 
Also, the resource allocation for each output changed less than 1 percent 
between the two runs. Therefore, the quality resource use function does not 
significantly affect the solution, at least within the range tested here. 
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7.7 Where Would New Resources Be Most Valuable? 

The resources in different categories, as explained earlier, are not 
interchangeable at present and do not have the same units (dollars and 
people). But, if the opportunity arose to reallocate an additional dollar or an 
additional person in any categories with similar units, where would it be the 
most valuable? For instance, would $1 investment in resource category X 
result in a greater increase in the objective function value than $1 investment 
in category Y? 

To answer this question, refer again to Appendix C, where the results of the 
model are presented. Two types of resources can be considered: those that 
have units of people and those that have units of dollars. I will consider each 
in turn. From the input of the model, the results indicate that in the people 
group the Co-Op category is the most valuable one, followed by FTE. This 
ranking was determined through the marginal values of FTE and Co-Op (see 
Section 7.3). However, after adding one person to the Co-Op category budget, 
the Co-Op category becomes as valuable as the FTE category. The second 
person added to the Co-Op category budget drops its marginal value to zero, 
meaning that additional Co-Op personnel will not increase the value of the 
objective function. At the same time, the value of FTE (marginal value) 
remains constant at 0.002 per FTE, which indicates that the objective function 
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value will increase 0.002 for each additional FTE. This conclusion is the result 
of three separate runs with the above data. 

In the group with dollar units, training is the most valuable resource, 
meaning that an additional dollar in the training funds will cause the most 
increase in the objective function value according to the marginal value of the 
resource. The marginal value of the training fund is 0.004 per dollar, which is 
double the marginal value of FTE. This result indicates that if FTE can be 
measured in dollar units like training, it would be much more beneficial to invest 
additional dollars in training than in FTE. 

7.8 Results of Survey on TSD’s Customer Satisfaction 

I now turn from a discussion of the model results to a review of the 
outcome of the customer satisfaction survey. This survey played a significant 
role in determining the quality of output and consequently in resources 
allocated (see Section 6.4). It is also appropriate to present these data here 
in order to give a comprehensive account of all the results of this study. 

Figure 7.12 represents the percentage of the respondents that gave various 
scores (1 to 5) for general satisfaction with each of the TSD divisions. Clearly, 
with the exception of FED, 50 to 60 percent of the informants gave 4 out of a 
possible 5 to each division indicating they were "satisfied" with the service 
received. Figure 7.13 compares the customers’ general satisfaction score 
with respect to other quality dimensions, showing that TSD needs to improve 
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Figure 7.12 General Satisfaction Score for Each Division. 


5 


lU 

a 

8 

(O 


)- 

o 

< 


w ft 

cr § 
aj £ 

s 5 

D 

O 


July 1393 


A.O 



GBCRAL 

SATISFACTION 



COMAJN I CAT ION RCSPOS I VEKESS FLEXIBILITY 

QUALITY DIMENSIONS 


3.S 



RELIABILITY 


Figure 7.13 Technical Services Directorate Customer Satisfaction. 
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communication with their customers. Also, the general satisfaction score is 
very much dependent on the communication score. This is evident from the 
regression analysis performed in Chapter 4. Three of the six divisions, TID, 
FOD, and FSD, have communication as the major quality dimension in 
determining general satisfaction. These three divisions serve most of the TSD 
customers, another reason why communication is a significant element of 
general satisfaction. 

Figure 7.14 represent the levels of customer general satisfaction with TSD, 
showing that 52 percent of the customers are satisfied with TSD services and 
30 percent are strongly satisfied. 
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Figure 7.14 General Satisfaction Score for TSD. 



Chapter 8 
Conclusions 

The need for this study is evident considering the $3.8 billion daily cost of 
operating the Federal Government [Department of Commerce, 1992] and its 
impact on the national debt. The constraints on the Federal budget, especially 
on the nonentitlement programs such as NASA, demand superior and less 
costly operations. We have to operate smartly and efficiently to meet our 
objectives with fewer resources. Every tool available must be used to help us 
accomplish our mission. This thesis focuses on creating a tool that optimizes 
resource allocation according to the quantity and quality of the outputs of the 
organization. The quality definition is based on the total quality management 
(TQM) philosophy. 

This tool is a mathematical model of the organization that is designed to 
capture the organization’s objectives, the decision-makers’ views, and the 
customers’ opinions on the outputs of the organization. Using nonlinear 
optimization software, the model answers the question, What is the best 
method, within the government regulations, of disbursing the limited resources 
to have the most satisfied customers? 

This study identified the input (resources) to the organization, developed 
metrics to measure the output of the organization, considered the customers’ 
opinions on the output, and packaged all the information into one nonlinear 
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programming model. Then, the model adjusted the resources assigned to each 
output systematically to find the combination of resources with the highest 
customer satisfaction. 

The solution is global optimum and unique, but it depends on the 
assumptions. The model can be a good and capable tool with which the 
decision-makers can examine all the scenarios before making any decision. 
But, like any other model, the accuracy of its output is directly related to the 
accuracy of the input. As TQM matures in government, more factual data will 
be available and the model will give more reliable information. The main 
contributions of this thesis are (1) developing metrics for the organization’s 
output, (2) estimating the impact of resource reallocation in real time, (3) 
providing a structured system for tracking and evaluating improvement in the 
organization, and (4) bringing customers’ issues and demands to the forefront. 

The challenge with implementing the model was in defining the outputs of 
the Technical Services Directorate (TSD) and the metrics to measure them. 
This difficulty can be attributed to the nature of NASA, where most of the 
products are one of a kind, and that this was the first attempt to quantify the 
outputs of TSD. 

The weights that were assigned to each output made a significant impact 
on the results of the model. Section 7.4 illustrates the results of equal weights 
versus assigned weights. The outputs and consequently the resources 
associated with the outputs increased and decreased with the weight of the 
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output. If the output was already at its upper bound even at a lower weight, 
the change in weight did not affect the output level. An example for this case 
is the FTE allocation for FPO, which is demonstrated in Figure 7.9. 

The resource use functions and the base year’s quality and quantity of 
outputs also played a significant role in the result of the model. To increase 
the quality and quantity levels S and X, decision-makers should strategically 
allocate scarce resources and use them more efficiently. In other words, 
decision-makers should do more with less to reduce RS and RX coefficients. 

The influence of budget size was tested by increasing the budget by 10 
percent and decreasing it by 10, 20, and 30 percent. The model became 
infeasible with a 30-percent budget cut and gave divisions near or above their 
current level of resources with a 1 0-percent increase in the budget. 

Given the limitations and assumptions of the model, the most obvious way 
to increase the quality of output in TSD is to improve communication with the 
customers. Communication received a low score from the customers, and it 
is one of the quality dimensions that has a significant impact on the overall 
general satisfaction with the divisions. Among resources, training (TRN) has 
the most "bang for the buck," followed by the number of FTEs. This means 
that, if permissible, an additional dollar should be invested in training rather 
than in an FTE. 

TSD will certainly benefit from involving the customers in every aspect of 
its operation and aligning the organization to the customers’ specific programs. 
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This additional scrutiny from the customers will adjust resource allocation to 
produce more satisfied customers, as demonstrated in allocation of the fund 
source 9 (FS9) resource. 

The mathematical model is designed to elevate the quality and quantity of 
all the directorate outputs. Therefore, it is possible that additional resources 
will be continuously allocated to the outputs that already have a high S value. 
The directorate budget is finite; hence, the additional funding that is 
appropriated to the product with a high S value reduces the funding to the 
product with a low S value. This part of the model does not allow the products 
with lower S values to receive additional funding to increase their customer 
satisfaction scores. Therefore, to give these products a chance, a policy might 
be suggested to improve less successful products, such as one-time capital 
investment. 

The model may be improved in the future by adding multiyear optimization 
for a period of 5 to 10 years, requiring additional information and strict policies 
on maintaining budgetary data. Also, future research to consider the 
relationship among various divisions’ inputs and outputs may be appropriate. 



Appendix A 

Quality Dimensions 

Parasuraman et al. [1985] have developed 10 service quality dimensions: 

(1 ) Reliability. — The firm performs the service right the first time and honors 
its promises, such as accurate billing and correct recordkeeping. 

(2) Responsiveness. — Employees are willing or ready to provide timely 
service. 

(3) Competence. — Employees have the required skills and knowledge to 
perform the service. 

(4) Access. — Access by telephone is easy, the waiting time to receive 
service is reasonable, and location and operating hours are convenient. 

(5) Courtesy. — Contact personnel are polite, respectful, considerate, and 
friendly. 

(6) Communication. — Customers are kept informed in language they can 
understand and are listened to. 

(7) Credibility. — Employees are trustworthy, believable, and honest and 
have the customers’ best interests at heart. 

(8) Security. — Customers are free from danger, risk, or doubt, and their 
confidentiality and privacy are respected. 

(9) Understanding. — The service provider makes the effort to understand 
the customers’ needs and requirements. 
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(10) Tangibles— Physical facilities, appearance of personnel, tools, or 
equipment used, and physical representation of service are satisfactory. 

Kennedy and Young [1989] developed six quality dimensions for service- 
oriented organizations. 

(1) Availability— the degree to which the customer can contact the service 
provider. 

(2) Responsiveness — the degree to which the service provider reacts to 
customers’ needs and requirements. 

(3) Convenience — the degree of ease with which the customers can interact 
with the service provider, which includes the facility location, office hours, 
meeting facilities, and effective communications skills. 

(4) Timeliness — the degree to which the total job is accomplished within the 

customer’s stated timeframe. 

(5) Completeness — the degree to which the total job is finished, including 
implementation, documentation, and follow-up. 

(6) Pleasantness — the degree to which the provider uses professional 
behavior and manner while working with the customer. 

Garvin [1988] concluded that service quality has eight dimensions: 

(1) Performance— the primary operating characteristics of a product or the 
speed or the absence of waiting time in the service area. 

(2) Features — secondary characteristics that supplement the product’s or 


service’s basic function. 
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(3) Reliability — the probability of a product’s malfunctioning within a specific 
time or consistency of service. 

(4) Conformance — the degree to which a product’s or service’s 
characteristics meet preestablished standards. 

(5) Durability— the amount of use one gets from a product before it breaks 
down and must be replaced. 

(6) Serviceability — the speed, courtesy, competence, and ease of repair. 

(7) Aesthetics — the way that a product looks, feels, and sounds and the 
general appearance of the output and operating environment. 

(8) Perceived quality — the product’s or service’s image and reputation, 
which are the perception of quality rather than the reality. 

Armistead [1989] claimed customer service is based on six dimensions: 

(1 ) Flexibility — coping with mistakes, either your own or those of customers, 
customizing the service, and introducing new services. 

(2) Absence of fault — correctness of information or advice, correctness of 
the specification, the physical items of the service package, and control 
procedures to measure and monitor the physical aspects of the service 
package. 

(3) A framework of time — the availability of the service, the responsiveness 
of the service organization, and the waiting or queuing time for the service. 
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(4) Style — the attitudes of service personnel, the accessibility of the entire 
organization to the customers, the ambiance of the service, and the perceived 
value of the service. 

(5) Steering — the clarity of the service in terms of where to go, what to do, 
who to see, and the sense that the customer is important and in control. 

(6) Safety — the customers feeling at ease with their position in the service 
organization. The factors are honesty, security, trust, and confidentiality. 

Nelson [1 990] established five quality dimensions in the health care industry 
that could be useful in this research because of its service orientation nature: 

(1 ) Access — appointment waiting time, telephone access, physical location, 
and operating hours. 

(2) Technical management— qualifications of staff, quality mechanism, and 
technical skills. 

(3) Interpersonal management — the way complaints or suggestions are 
handled, the amount of time spent with the client, and the courtesy of 
employees. 

(4) Continuity of care — consistent attention to customers and the customer 
knowing who should address his or her specific problem. 

(5) General satisfaction— the general perspective the customer has in 
dealing with this organization. 

Hyde [1992] explained seven quality dimensions for TQM in the public 


sector: 
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(1) Reaction time — responsiveness to problems or emergencies. 

(2) Timeliness — commitment to scheduled compliance. 

(3) Commitment to budget and cost control. 

(4) Defect rate — the rate of error or compliance. 

(5) Professionalism — the work attitude and commitment to quality. 

(6) Service attitude — identifying with the customers’ needs. 

(7) Follow-up — responding to the customers’ complaints and rectifying the 
service provider’s mistakes. 


Appendix B 

Analytic Hierarchy Process 

In AHP, the problem is decomposed into its elements and organized in a 
multilevel structure where the main objective occupies the top level, which is 
called the focus. Each intermediate level is made of several elements, called 
criteria, that are compared with one another against an element at the next 
higher level. The lowest level of the structure comprises alternatives that are 
under consideration [Saaty 1 982]. 

With this structure in mind, the principle of discrimination and comparative 
judgments is utilized to establish priorities among the criteria and alternatives. 
The method of applying this principle is the pairwise comparisons process. The 
best way to explain that is to demonstrate the process for a typical hierarchical 
structure shown in Table B.l. 

Table B.l. Typical Hierarchical Structure 

Level 1 : Focus A 

Level 2: Criteria B1 B2 B3 

Level 3: Criteria Cl C2 C3 C4 C5 

Level 4: Alternatives D1 D2 D3 

The process begins from the lowest level of the AHP structure, which is 
level 4, the alternatives. Every element of this level is compared with the rest 
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of the elements of the same level against all the elements from the next higher 
level. In other words, the elements on any level (except the lowest level) 
become categories of comparison for the elements on the next lower level 
[Saaty 1982, Saaty and Keam 1985]. 

Analytically, the comparison is performed in a matrix form. The matrix is 
constructed from the ratios of the relative importance of the elements of each 
level with respect to a criterion provided by the elements in the next higher 
level. The relative importance of an element is expressed by a numerical value 
from 1 through 9. The scale of nine units is based purely on experience, and 
it is proven to be adequate to portray the discrimination among the elements 
[Saaty 1982]. 

As an example, consider the typical hierarchical structure presented in 
Table B.1 . The comparison matrix for the fourth level is formed from the ratios 
of the impact of each element (D1 , D2, D3) to an element at the third level (Cl 
to C5). Because there are five elements in the third level, there will be five 
comparison matrices for the fourth level [Saaty and Kearn 1985]. 

The relative importance of an element is designated by the decision-maker 
and reflects his or her judgments and personal views on the subject element. 
The key function of the hierarchical modeling is to translate human values into 
a mathematical format where it can be studied and optimized by applying all 
the mathematical tools and capabilities. 
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The last step of AHP is to solve the pairwise matrices and pull all the 
judgments together into a single number that demonstrates the priority of each 
element with respect to the others. To illustrate this process, consider the 
example in Table B.1 . The comparative matrix for the lowest level of the model 
comprises the criteria from the third level listed in the upper left-hand corner. 
The alternatives, D1 to D3, are listed in the top row and the left-hand column. 
Table B.2 demonstrates a typical three-element comparative matrix [Saaty 
1982]. 

Table B.2. Comparative Matrix 


Cl 

D1 

D2 

D3 

D1 

1 

1/m 

1/n 

D2 

m 

1 

1/p 

D3 

n 

P 

1 


The diagonal entries are always 1 because an element is compared with 
itself. An entry such as m is the ratio of the relative importance of D2 over D1 
with respect to Cl . For example, if Cl is "comfort' and D2 is a type of car, say 
Lexus, and D1 is another type, say Yugo, then D2 is m times more comfortable 
than D1 . This is the reason that the entries across the diagonal are reciprocals 

as indicated. 

Once the matrix is filled, in order to synthesize the judgments, the matrix 
is normalized by dividing each entry by the total value of its column. In this 
step, entry 1 will be transformed to 1/(1 + m + n). The rows of the normalized 
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matrix will be averaged to produce the overall ranking of the elements. This 
process will continue until the overall importance of every element of the 
hierarchical model is developed [Saaty 1982]. 

In the mathematical context, the comparison matrix, which is referred to as 
C hereafter, is a single-rank matrix because every row is a multiplier of the first 
row. 



c, 

c 2 

c„ 

C, 

w,/w, 

W^Wg 

“A 

O 

II 

o 

. to 

Wj/W, 

Wg/W 2 

Wj/W n 

c„ 

wyw, 

w„/w 2 ... 

w„/w„ 


All entries of this matrix are positive and the matrix has the reciprocal property 
where an entry Cy is equal to 1 /c j( . It is interesting to note that if C is multiplied 
by the vector w = (w,, ..., w n ), the resulting vector is nw. 

Cw = nw (B-1) 

From the above equation, (C - ln)w = 0 is ascertained, where w is the only 
unknown. The nonzero value for w is possible only if C - In = 0, meaning that 
n is the eigenvalue of C. Because the rank of C is 1, there is only one 
eigenvalue that is nonzero and equal to n. This eigenvalue is called The 
priority vector is any column of the matrix C with a different constant multiplier. 
It is desired to normalize the solution so that its components sum to unity. The 
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normalized vector w creates a unique solution no matter which column is used. 
The final priority vector that is developed through the above procedure is 
actually this normalized solution. 

According to Saaty [1977], a reciprocal matrix with positive entries is 
consistent if and only if = n, where n is the number of objects that are 
compared. A measure for consistency is established to be CR = (X^ - n)/ 

(n - 1). Equation (B-1) can be rewritten as 

Cw = X^w (B-2) 

and X max is approximated by using equation (B-2), the normalized vector w, and 
the matrix C. 

Now that X^ is computed, the value of CR is known. Saaty and Mariano 
[1979] found mean inconsistency for samples of 500 random filled matrices of 
each size from 2-by-2 to 10-by-10 matrices. The numerical judgments were 

taken at random from the scale 1/9, 1/8, 1/7, ..., 1/2 1, 2, ..., 9. Then, 

using a reciprocal matrix would give the following average consistencies for 
different-order random matrices: 

Size of matrix 12345 6 7 8 9 10 

Random consistency 0 0 .58 .9 1.12 1.24 1.32 1.41 1.45 1.49 

A value of 1 0 percent or less for the ratio of CR over the appropriate random 
consistency is recommended for a consistent judgment. 
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Tables B.3 to B.19 represent the first AHP that was conducted with the 
decision-makers. 

Table B.3 First Hierarchy for TSD Outputs 
FOCUS: TSD MISSION R&D TEST 

ATTRIBUTES: HWD FAC H/S/E ENERGY 

DIVISIONS: FPO OEP TID FOD FSD FED 


Table B.4 Initial Weight Ratio (Column/Row) for Hardware 


HWD 

FPO 

OEP 

FPO 

1.00 

0.33 

OEP 

3.00 

1.00 

TID 

7.00 

5.00 

FOD 

3.00 

1.00 

FSD 

9.00 

8.00 

FED 

3.00 

0.33 

TOTAL 

26.00 

15.67 


Cl = 0.08 


TID 

FOD 

FSD 

FED 

0.14 

0.33 

0.11 

0.33 

0.20 

1.00 

0.13 

3.00 

1.00 

7.00 

0.20 

7.00 

0.14 

1.00 

0.11 

1.00 

5.00 

9.00 

1.00 

9.00 

0.14 

1.00 

0.11 

1.00 

6.63 

19.33 

1.66 

21.33 


Table B.5 Normalized Matrix With Eigenvector for Hardware 


1 

HWD 

FPO 

OEP 

TID 

FOD 

FSD 

FED 

PRI 

VECTOR 

(PV) 

FPO 

0.04 

0.02 

0.02 

0.02 

0.07 

0.02 

0.03 

OEP 

0.12 

0.06 

0.03 

0.05 

0.08 

0.14 

0.08 

TID 

0.27 

0.32 

0.15 

0.36 

0.12 

0.33 

0.26 

FOD 

0.12 

0.06 

0.02 

0.05 

0.07 

0.05 

0.06 

FSD 

0.35 

0.51 

0.75 

0.47 

0.60 

0.42 

0.52 

FED 

0.12 

0.02 

0.02 

0.05 

0.07 

0.05 

0.05 

TOTAL 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 
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Table B.6 Pairwise Comparison Matrix for Hardware 


HWD 

FPO'PV 

OEP’PV 

TID*PV 

FOD'PV 

FSD*PV 

FED*PV 

ROW 

TOTAL 

RT/PV 

FPO 

0.03 

0.03 

0.04 

0.02 

0.06 

0.02 

0.19 

6.27 

OEP 

0.09 

0.08 

0.05 

0.06 

0.06 

0.16 

0.51 

6.41 

TID 

0.21 

0.40 

0.26 

0.43 

0.10 

0.38 

1.78 

6.87 

FOD 

0.09 

0.08 

0.04 

0.06 

0.06 

0.05 

0.38 

6.21 

FSD 

0.27 

0.64 

1.29 

0.55 

0.52 

0.49 

3.75 

7.26 

FED 

0.09 

0.03 

0.04 

0.06 

0.06 

0.05 

0.33 

6.05 


AVR. : 6.51 



2.71 20.53 24.33 5.73 28.00 3.43 

TOTAL 


Cl = 0.10 
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Table B.8 Normalized Matrix With Eigenvector for Facilities 


FAC 

FPO 

OEP 

TID 

FOD 

FSD 

FED 

PRI 

VECTOR 

FPO 

0.37 

0.34 

0.33 

0.52 

0.32 

0.29 

0.36 

OEP 

0.05 

0.05 

0.12 

0.03 

0.18 

0.04 

0.08 

T1D 

0.05 

0.02 

0.04 

0.03 

0.11 

0.04 

0.19 

FOD 

0.12 

0.24 

0.21 

0.17 

0.11 

0.29 

0.19 

FSD 

0.04 

0.01 

0.01 

0.06 

0.04 

0.04 

0.03 

FED 

0.37 

0.34 

0.29 

0.17 

0.25 

0.29 

0.29 

TOTAL 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 


Table B.9 Pairwise Comparison Matrix for Facilities 


FAC 

FPO*PV 

OEP*PV 

T!D*PV 

FOD*PV 

FSD*PV 

FED*PV 

ROW 

TOTAL 

RT/PV 

FPO 

0.36 

0.56 

0.38 

0.57 

0.30 

0.29 

2.46 

6.80 

OEP 

0.05 

0.08 

0.14 

0.04 

0.17 

0.04 

0.52 

6.51 

TID 

0.05 

0.03 

0.05 

0.04 

0.10 

0.04 

0.30 

6.24 

FOD 

0.12 

0.40 

0.24 

0.19 

0.10 

0.29 

1.34 

7.00 

FSD 

0.04 

0.02 

0.02 

0.06 

0.03 

0.04 

0.21 

6.30 

FED 

0.36 

0.56 

0.33 

0.19 

0.23 

0.29 

1.97 

6.89 


AVR.: 6.62 


Table B.10 Initial Weight Ratio (Column/Row) 
for Health and Safe Environment 


H/S/E 

FPO 

OEP 

FPO 

1.00 

0.13 

OEP 

8.00 

1.00 

TID 

7.00 

0.25 

FOD 

7.00 

0.25 

FSD 

5.00 

0.17 

FED 

7.00 

0.25 

TOTAL 

35.00 

2.04 


Cl = 0.11 


TID 

FOD 

FSD 

FED 

0.14 

0.14 

0.20 

0.14 

4.00 

4.00 

6.00 

4.00 

1.00 

1.00 

3.00 

3.00 

1.00 

1.00 

4.00 

2.00 

0.33 

0.25 

1.00 

3.00 

0.33 

0.50 

0.33 

1.00 

6.81 

6.89 

14.53 

13.14 


Table B.11 Normalized Matrix With Eigenvector 


for Health and Safe Environment 


H/S/E 

FPO 

OEP 

TID 

FPO 

0.03 

0.06 

0.02 

OEP 

0.23 

0.49 

0.59 

TID 

0.20 

0.12 

0.15 

FOD 

0.20 

0.12 

0.15 

FSD 

0.14 

0.08 

0.05 

FED 

0.20 

0.12 

0.05 

TOTAL 

1.00 

1.00 

1.00 


PRI 


FOD 

FSD 

FED 

VECTOR 

0.02 

0.01 

0.01 

0.03 

0.58 

0.41 

0.30 

0.43 

0.15 

0.21 

0.23 

0.17 

0.15 

0.28 

0.15 

0.17 

0.04 

0.07 

0.23 

0.10 

0.07 

0.02 

0.08 

0.09 

1.00 

1.00 

1.00 

1.00 
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Table B.12 Pairwise Comparison Matrix for Health and Safe Environment 


H/S/E 

fpctpv 

OEP*PV 

T!D*PV 

FOD'PV 

FSD'PV 

FED*PV 

ROW 

TOTAL 

RT/PV 

FPO 

0.03 

0.05 

0.02 

0.02 

0.02 

0.01 

0.16 

6.27 

OEP 

0.21 

0.43 

0.70 

0.69 

0.61 

0.36 

3.00 

6.93 

TID 

0.18 

0.11 

0.17 

0.17 

0.30 

0.27 

1.21 

6.94 

FOD 

0.18 

0.11 

0.17 

0.17 

0.40 

0.18 

1.22 

7.05 

FSD 

0.13 

0.07 

0.06 

0.04 

0.10 

0.27 

0.68 

6.69 

FED 

0.18 

0.11 

0.06 

0.09 

0.03 

0.09 

0.56 

6.19 








AVR.: 

6.68 


Table B.13 Initial Weight Ratio (Column/Row) for Energy 

ENERGY 

FPO 

OEP 

T!D 

FOD 

FSD 

FED 

FPO 

1.00 

3.00 

4.00 

0.17 

4.00 

1.00 

OEP 

0.33 

1.00 

4.00 

0.33 

4.00 

3.00 

TID 

0.25 

0.25 

1.00 

0.14 

3.00 

0.33 

FOD 

6.00 

3.00 

7.00 

1.00 

7.00 

3.00 

FSD 

0.25 

0.25 

0.33 

0.14 

1.00 

0.17 

FED 

1.00 

0.33 

3.00 

0.33 

6.00 

1.00 

TOTAL 

8.83 

7.83 

19.33 

2.12 

25.00 

8.50 


Cl = 0.11 
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Table B.14 Normalized Matrix With Eigenvector 


ENERGY 

FPO 

OEP 

TID 

FOD 

FSD 

FED 

PRI 

VECTOR 

FFO 

0.11 

0.38 

0.21 

0.08 

0.16 

0.12 

0.18 

OEP 

0.04 

0.13 

0.21 

0.16 

0.16 

0.35 

0.17 

T1D 

0.03 

0.03 

0.05 

0.07 

0.12 

0.04 

0.06 

FOD 

0.68 

0.38 

0.36 

0.47 

0.28 

0.35 

0.42 

FSD 

0.03 

0.03 

0.02 

0.07 

0.04 

0.02 

0.03 

FED 

0.11 

0.04 

0.16 

0.16 

0.24 

0.12 

0.14 

TOTAL 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 


Table B.15 Pairwise Comparison Matrix tor Energy 


ENERGY 

FPO*PV 

OEP*PV 

T1D*PV 

FOD*PV 

FSD*PV 

FED*PV 

ROW 

TOTAL 

RT/PV 

FPO 

0.18 

0.52 

0.23 

0.07 

0.14 

0.14 

1.27 

7.18 

OEP 

0.06 

0.17 

0.23 

0.14 

0.14 

0.41 

1.15 

6.61 

TID 

0.04 

0.04 

0.06 

0.06 

0.10 

0.05 

0.35 

6.24 

FOD 

1.06 

0.52 

0.40 

0.42 

0.24 

0.41 

3.05 

7.23 

FSD 

0.04 

0.04 

0.02 

0.06 

0.03 

0.02 

0.22 

6.56 

FED 

0.18 

0.06 

0.17 

0.14 

0.20 

0.14 

0.89 

6.44 


AVR.: 6.71 

Table B.16 Initial Weight Ratio (Column/Row) 
for R&D Test 


R&D 

TEST 

HWD 

FAC 

H/S/E/ 

ENERGY 

HWD 

1.00 

8.00 

5.00 

7.00 

FAC 

0.13 

1.00 

1.00 

1.00 

H/S/E/ 

0.20 

1.00 

1.00 

4.00 

ENERGY 

0.14 

1.00 

0.25 

1.00 

TOTAL 

1.47 

11.00 

7.25 

13.00 


Cl = 0.08 
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Table B.17 Normalized Matrix With Eigenvector 
for R&D Test 


R&D 

TEST 

HWD 

FAC 

H/S/E/ 

ENERGY 

PRI 

VECTOR 

HWD 

0.68 

0.73 

0.69 

0.54 

0.66 

FAC 

0.09 

0.09 

0.14 

0.08 

0.10 

H/S/E/ 

0.14 

0.09 

0.14 

0.31 

0.17 

ENERGY 

0.10 

0.09 

0.03 

0.08 

0.07 

TOTAL 

1.00 

1.00 

1.00 

1.00 

1.00 


Table B.18 Pairwise Comparison Matrix for R&D Test 


R&D 

TEST 

HWD* 

PV 

PAC* 

PV 

H/S/E/*PV 

ENERGY* 

PV 

ROW 

TOTAL 

RT/PV 

HWD 

0.66 

0.78 

0.84 

0.52 

2.81 

4.26 

FAC 

0.08 

0.10 

0.17 

0.07 

0.42 

4.33 

H/S/E/ 

0.13 

0.10 

0.17 

0.30 

0.70 

4.15 

ENERGY 

0.09 

0.10 

0.04 

0.07 

0.31 

4.12 


AVR.: 


4.21 
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APPENDIX C : RESULTS OF THE MODEL EXECUTION 

GENERAL ALGEBRAIC MODELING SYSTEM 
MODEL STATISTICS SOLVE ALLTSD USING NLP FROM LINE 773 


MODEL STATISTICS 


BLOCKS OF EQUATIONS 

53 

SINGLE EQUATIONS 

53 

BLOCKS OF VARIABLES 

107 

SINGLE VARIABLES 

107 

NON ZERO ELEMENTS 

391 

NON LINEAR N-Z 

97 

DERIVATIVE POOL 
CODE LENGTH 

65 

1149 

CONSTANT POOL 

88 

GENERATION TIME 

= 

0.374 MINUTES 


EXECUTION TIME 

= 

0.492 MINUTES 
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GENERAL ALGEBRAIC MODELING SYSTEM 
SOLUTION REPORT SOLVE ALLTSD USING NLP FROM LINE 773 


SOLVE 

MODEL ALLTSD 
TYPE NLP 
SOLVER MINOS5 

**** SOLVER STATUS 1 

**** MODEL STATUS 2 

**** OBJECTIVE VALUE 

RESOURCE USAGE, LIMIT 
ITERATION COUNT, LIMIT 
EVALUATION ERRORS 

MINOS 5.2 


SUMMARY 

OBJECTIVE Z 
DIRECTION MAXIMIZE 
FROM LINE 773 

NORMAL COMPLETION 
LOCALLY OPTIMAL 

3.5288 

16.087 1000.000 

567 1000 

0 0 

(Mar 1988) 


B. A. Murtagh, University of New South Wales 
and 

P. E. Gill, W. Murray, M. A. Saunders and M. H. Wright 
Systems Optimization Laboratory, Stanford University. 


MINOS 5.2 (Mar 1988) 


OPTIONS file 


BEGIN GAMS/MINOS OPTIONS 

MAJOR ITERATIONS LIMIT 50.0 

MINOR ITERATIONS LIMIT 200 

END GAMS/MINOS OPTIONS 

WORK SPACE NEEDED (ESTIMATE) — 7503 WORDS. 

WORK SPACE AVAILABLE — 8100 WORDS. 


EXIT — OPTIMAL SOLUTION FOUND 


MAJOR ITNS, LIMIT 10 
FUNOBJ , FUNCON CALLS 1349 
SUPERBASICS 6 
INTERPRETER USAGE 5.62 
NORM RG / NORM PI 4.186E-07 


50 

1349 
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GENERAL ALGEBRAIC MODELING SYSTEM 
SOLUTION REPORT SOLVE ALLTSD USING NLP FROM LINE 773 


LOWER 

EQU FTEFPO 

EQU FTEOEP 

EQU FTETID 

EQU FTEFOD 

EQU FTEFSD 

EQU FTEFED 

EQU SSCFPO 

EQU SSCOEP 

EQU SSCTID 

EQU SSCFOD 

EQU SSCFSD 

EQU SSCFED 

EQU ROSFPO 

EQU ROSOEP 

EQU ROSTID 

EQU ROSFOD 

EQU ROSFSD 

EQU ROSFED 

EQU FS9FP0 

EQU FS90EP 

EQU FS9FOD 

EQU FS9FSD 

EQU FS9FED 

EQU COPOEP 

EQU COPTID 

EQU COPFSD 

EQU TRNFPO 

EQU TRNOEP 

EQU TRNTID 

EQU TRNFOD 

EQU TRNFSD 

EQU TRNFED 

EQU TRVFPO 

EQU TRVOEP 

EQU TRVTID 

EQU TRVFOD 

EQU TRVFSD 

EQU TRVFED 

EQU EQPFPO 

EQU EQPOEP 

EQU EQPTID 

EQU EQPFOD 

EQU EQPFSD 

EQU EQPFED 


LEVEL UPPER MARGINAL 

- 0.002 

- 0.002 

- 0.002 

- 0.002 

- 0.002 

- 0.002 

EPS 

EPS 

EPS 

EPS 

EPS 

EPS 

-5.427E-6 

-5.427E-6 

-5.427E-6 

-5.427E-6 

-5.427E-6 

-5.427E-6 

EPS 

EPS 

EPS 

EPS 

EPS 

-0.003 

-0.003 

-0.003 

-0.004 

-0.004 

-0.004 

-0.004 

-0.004 

-0.004 

EPS 

EPS 

EPS 

EPS 

EPS 

EPS 

EPS 

EPS 

EPS 

EPS 

EPS 

EPS 
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EQU FTECON 

-INF 

707.000 

707.000 

0.002 

EQU SSCCON 

-INF 

555.733 

574.000 

EPS 

EQU ROSCON 

-INF 

15969.000 

15969.000 

5.4272E-6 

EQU FS9C0N 

-INF 

14050.149 

15104.000 

EPS 

EQU COPCON 

-INF 

44.000 

44.000 

0.003 

EQU TRNCON 

-INF 

193.000 

193.000 

0.004 

EQU TRVCON 

-INF 

187.280 

196.000 

EPS 

EQU EQPCON 

-INF 

1742.894 

1806.000 

EPS 

EQU OBJECT 

• 

• 

• 

1.000 
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GENERAL ALGEBRAIC MODELING SYSTEM 
SOLUTION REPORT SOLVE ALLTSD USING NLP FROM LINE 773 


FTEFPO 

FTEOEP 

FTETID 

FTEFOD 

FTEFSD 

FTEFED 

SSCFPO 

SSCOEP 

SSCTID 

SSCFOD 

SSCFSD 

SSCFED 

ROSFPO 

ROSOEP 

ROST ID 

ROSFOD 

ROSFSD 

ROSFED 

FS9FPO 

FS90EP 

FS9F0D 

FS9FSD 

FS9FED 

COPOEP 

COPTID 

COPFSD 

TRNFPO 

TRNOEP 

TRNTID 

TRNFOD 

TRNFSD 

TRNFED 

TRVFPO 

TRVOEP 

TRVTID 

TRVFOD 

TRVFSD 

TRVFED 

EQPFPO 

EQPOEP 

EQPTID 

EQPFOD 

EQPFSD 

EQPFED 

FTECON 

SSCCON 

ROSCON 


FTE CONSTRAINT 
SSC CONSTRAINT 
ROS CONSTRAINT 
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FS9C0N FS9 CONSTRAINT 

COPCON CO-OP CONSTRAINT 

TRNCON TRAINING COSTRAINT 

TRVCON TRAVEL CONSTRAINT 

EQPCON EQUIPMENT CONSTRAINT 

OBJECT OBJECTIVE VALUE 
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LGEBRAIC MODELING SYSTEM 
SOLVE ALLTSD USING NLP FROM LINE 773 


GENERAL A 
SOLUTION REPORT 


VAR SI 

VAR S2 

VAR S3 

VAR S4 

VAR S5 

VAR S6 

VAR Xll 

VAR X12 

VAR X13 

VAR X14 

VAR X15 

VAR X16 

VAR X21A 

VAR X21B 

VAR X21C 

VAR X21D 

VAR X22A 

VAR X22B 

VAR X22C 

VAR X22D 

VAR X23A 

VAR X23B 

VAR X23C 

VAR X24A 

VAR X24B 

VAR X24C 

VAR X25 

VAR X31 

VAR X32 

VAR X33 

VAR X34 

VAR X35 

VAR X44A 

VAR X44B 

VAR X44C 

VAR X44D 

VAR X45 

VAR X46A 

VAR X46B 

VAR X46C 

VAR X46D 

VAR X46E 

VAR X46F 

VAR X46G 


LOWER LEVEL 


UPPER MARGINAL 


1.000 

4.696 

5.000 


1.000 

4.487 

5.000 


1.000 

4.405 

5.000 


1.000 

3.096 

5.000 


1.000 

4.596 

5.000 


1.000 

2.177 

5.000 



20.000 

20.000 

0.005 


40.000 

40.000 

0.002 


1 . 5000E+5 

1.5000E+5 

EPS 


100.000 

100.000 

2.6467E-4 


100.000 

100.000 

1.3318E-4 

7.000 

100.000 

100.000 

1. 1889E-4 


500.000 

500.000 

6.6142E-5 

12.000 

60.000 

60.000 

5.4371E-4 


7000.000 

7000.000 

3.4467E-6 

8.000 

120.000 

120.000 

2.6683E-4 


3.5000E+5 7.5520E+5 2.0000E+6 EPS 

24.000 24.000 0.001 

1000.000 2000.000 2000.000 2.0026E-5 

40.000 40.000 0.001 

10.000 10.000 0.001 

10.000 20.000 20.000 7.0303E-4 

1000.000 1200.000 1200.000 1.2558E-5 

500.000 1200.000 1200.000 2.4018E-5 

20.000 20.000 0.002 

180.000 180.000 1.6848E-4 

500.000 1410.147 4000.000 

25000.000 25000.000 33000.000 -2.365E-6 
1.2500E+5 1.6500E+5 1.6500E+5 1.0713E-6 
1.5000E+5 1.9800E+5 1.9800E+5 1.2776E-6 
1.2500E+5 1.6500E+5 1.6500E+5 1.0830E-6 

75000.000 99000.000 99000.000 EPS 

2.000 15.000 15.000 2.0452E-4 

12.000 12.000 120.000 -1.044E-4 

40.000 -9.955E-4 
25oloOO 250.000 5000.000 -2.506E-6 

1.2500E+6 1.5896E+6 1.7880E+6 EPS 

80000.000 80000.000 2.4000E+5 EPS 

10000.000 10000.000 EPS 

1.0000E+5 EPS 

1.5000E+5 1.5000E+5 EPS 

75000.000 EPS 

500.000 -2.227E-5 

2080.000 2080.000 1.0000E+5 EPS 
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VAR 

X46H 

• 

• 

50000.000 

EPS 

VAR 

X47 

8000.000 

10000.000 

10000.000 

3.9372E-6 

VAR 

X410 

• 

• 

84651.000 

EPS 

VAR 

X411 

• 

1.0640E+5 

1.0640E+5 

EPS 

VAR 

X412A 

100.000 

220.000 

220.000 

6.2978E-5 

VAR 

X412B 

33.000 

100.000 

100.000 

4.3615E-5 

VAR 

X413 

4.000 

5.425 

6.000 

• 

VAR 

X414 

# 

• 

10.000 

-0.004 

VAR 

X51 

6.2850E+6 

6.2850E+6 

7. 1660E+6 

EPS 
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GENERAL A 

L G E B R A 

I C MO 

D E L I N 

G SYS 

SOLUTION REPORT 

SOLVE ALLTSD USING NLP FROM LINE 773 


LOWER 

LEVEL 

UPPER 

MARGINAL 

VAR X52 

5906.000 

6469.000 

6469.000 

3. 1099E-5 

VAR X53 

35859.000 

44465.000 

44465.000 

3.1518E-6 

VAR X54 

46080.000 

46080.000 

6.2208E+5 

-1.283E-6 

VAR X55 

10492.000 

13954.000 

13954.000 

8.2037E-6 

VAR X56 

28421.000 

34390.000 

34390.000 

EPS 

VAR X61 

350.000 

500.000 

500.000 

6.3847E-5 

VAR X62 

30.000 

70.000 

70.000 

2.9608E-4 

VAR X63 

1000.000 

1000.000 

1500.000 

-4.314E-5 

VAR X64 

40.000 

52.000 

52.000 

1.8845E-4 

VAR FTE1 

-INF 

31.227 

+INF 

• 

VAR FTE2 

-INF 

34.223 

+INF 


VAR FTE3 

-INF 

343.074 

+INF 


VAR FTE4 

-INF 

84.246 

+INF 


VAR FTE5 

-INF 

139.840 

+INF 


VAR FTE6 

-INF 

74.390 

+INF 


VAR SSC1 

-INF 

4.285 

+INF 


VAR SSC2 

-INF 

26.399 

+INF 


VAR SSC3 

-INF 

69.260 

+INF 


VAR SSC4 

-INF 

391.193 

+INF 


VAR SSC5 

-INF 

40.251 

+INF 


VAR SSC6 

-INF 

24.345 

+INF 


VAR ROS1 

-INF 

2984.618 

+INF 


VAR ROS2 

-INF 

3154.934 

+INF 


VAR ROS3 

-INF 

901.449 

+INF 


VAR R0S4 

-INF 

8398.092 

+INF 


VAR ROS5 

-INF 

26.803 

+INF 


VAR ROS6 

-INF 

503.104 

+INF 


VAR FS91 

-INF 

886.154 

+INF 


VAR FS92 

-INF 

536.884 

+INF 


VAR FS94 

-INF 

11283.436 

+INF 


VAR FS95 

-INF 

333.742 

+INF 


VAR FS96 

-INF 

1009.933 

+ INF 


VAR COP2 

-INF 

3.033 

+INF 


VAR COP3 

-INF 

35.247 

+INF 


VAR COP5 

-INF 

5.720 

+INF 


VAR TRN1 

-INF 

11.374 

+INF 


VAR TRN2 

-INF 

21.029 

+INF 


VAR TRN3 

-INF 

76.128 

+INF 


VAR TRN4 

-INF 

27.734 

+INF 


VAR TRN5 

-INF 

40.868 

+INF 


VAR TRN6 

-INF 

15.866 

+INF 


VAR TRV1 

-INF 

10.461 

+INF 


VAR TRV2 

-INF 

31.754 

+INF 


VAR TRV3 

-INF 

58.729 

+INF 

• 


T E M 
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VAR TRV4 

VAR TRV5 

VAR TRV6 

VAR EQP1 

VAR EQP2 

VAR EQP3 

VAR EQP4 

VAR EQP5 

VAR EQP6 


-INF 

29.669 

-INF 

35.488 

-INF 

21.180 

-INF 

84.213 

-INF 

150.411 

-INF 

277.464 

-INF 

224.076 

-INF 

799.528 

-INF 

207.201 


+INF 

+INF 

+INF 

+INF 

+INF 

+INF 

+INF 

+INF 

+INF 
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GENERAL ALGEBRAIC MODELING SYSTEM 
SOLUTION REPORT SOLVE ALLTSD USING NLP FROM LINE 773 


LOWER LEVEL UPPER MARGINAL 
VAR Z -INF 3.529 +INF 


**** REPORT SUMMARY : 0 NONOPT 

0 INFEASIBLE 
0 UNBOUNDED 
0 ERRORS 


**** FILE SUMMARY 

INPUT A:\CTSD-ML.GMS 
OUTPUT A:\CTSD-ML.LST 

EXECUTION TIME = 0.166 MINUTES 



Appendix D 

TSD Functions and Resources 
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Dear Customer: 

You have been identified as one of the Facilities Planning Office (FPO) customers. Your candid and honest 
opinion is needed to improve and maintain the quality of our services. Please take a moment and complete the 
enclosed survey and return it to me by Friday. April 23, 1993 at Mail Stop 501-8. If you wish to remain anonymous, 
please remove this sheet and mail the remainder. If you have any questions please call me at 3-6753. 

Thank You, 


James Afarin 



CUSTOMER OPINION SURVEY 


Thank you for participating in this survey. To better serve you, we would like to know your opinion of the quality 
of service provided by the Facilities Planning Office (FPO). FPO is charged with facilities planning and utilization, 
energy management and contract management. The products and services of FPO are: Annual Construction of 
Facilities (CoF) budget, center funded projects implementation, space management, real property management, 
energy management and support service contract management. There are no right or wrong answers, however, 
your honest opinions are very important. If you wish to make additional comments, please use the space provided 
at the end of the survey. Pease evaluate these statements using the following scale. 


1- 1 Strongly Disagree with this statement (SD). 

2- 1 Disagree with this statement (D). 

3- 1 Neither agree nor disagree with this statement (N). 


4- 1 Agree with this statement (A). 

5- 1 Strongly Agree with this statement (SA). 
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FPO QUESTIONNAIRE 


General Satisfaction 

1 . I am generally satisfied with the service I receive from FPO. 

Convenience 

2. FPO personnel are easily accessible at appropriate time to satisfy my requirements. 

3. I know the right person in FPO that I should contact for my problem. 

Communication 

4. FPO keeps me informed on the status of my project 

5. I receive clear and accurate explanation on the procedures and limitations of the FPO 
services. 

6. I am satisfied with the professionalism of contact personnel at FPO. 

7. I believe that FPO personnel understand my specific needs. 

8. I believe that FPO personnel have my best interest in heart. 

Responsiveness 

9. FPO responds to my complaints and rectifies my problems. 

10. I am satisfied with the time that is required to return my phone cal. 

1 1 . I am satisfied with the time that is required to respond to my inquiries in: 

a. CoF budget process 

b. Center funded projects implementation 

c. Space management 

d. Calspan contract management 

e. Real property management 

f. Energy management 


SD D N A SA 
1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
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Flexibility 

12. I am satisfied with the ability of FPO to cope with mistakes and unforeseen problems. 

13. lam satisfied with the management of FPO in assigning the right person for the job. 
Reliability - 

14. FPO performs the service right the first time. 

15. FPO performs its services at the time it promises to do so. 

Demand 

16. FPO should increase its support in the following areas: 

a. CoF budget process 

b. Center funded projects implementation 

c. Space management 

d. Calspan contract management 

e. Real property management 

f. Energy management 


1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 


1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
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17. What specific thing can FPO do to increase your satisfaction with this organization? 


18. What aspect of service from FPO is not covered in this questionnaire? 


Additional Comments: 



Dear Customer: 


You have been identified as one of the Office of Environmental Programs (OEP) customers. Your candid and 
honest opinion is needed to improve and maintain the quality of our services. Please take a moment and complete 
the enclosed survey and return it to me by Friday, April 16, 1993 at Mail Stop 501-8. If you wish to remain 
anonymous, please remove this sheet and mail the remainder. If you have any questions please call me at 3-6753. 

Thank You, 


James Afarin 



244 

CUSTOMER OPINION SURVEY 


Thank you for participating in this survey. To better serve you, we would like to know your opinion of the quality 
of service provided by the Office of Environmental Programs (OEP). OEP serves as a consultant to the staff in 
environmental compliance. The products and services of OEP include: Industrial hygiene, environmental 
compliance, hazardous chemicals handling, health physics and chemical sampling and analysis. There are no right 
or wrong answers; however, your honest opinions are very important. If you wish to make additional comments, 
please use the space provided at the end of the survey. Please evaluate these statements using the following scale: 

1- 1 Strongly Disagree with this statement (SD). 4- 1 Agree with this statement (A). 

2- 1 Disagree with this statement (D). 5- 1 Strongly Agree with this statement (SA). 

3- 1 Neither agree nor disagree with this statement (N). 

Please identify the branch that you deal with the most. If you deal with more than one branch, please assign a 
percentage of interaction to each branch. 


Orq. Code 

Branch Name 

Branch Chief 

7021 

Industrial Hygiene Office 

Blotzer 

7022 

Environmental Compliance Office 

Watson 

7023 

Hazardous Chemicals Office 

Dominguez 

7024 

Health Physics Office 

Smith 

7025 

Chemical Sampling and Analysis Office 

Street 
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PEP QUESTIONNAIRE 


1 . 


During the last six months, I received services from OEP 


times. 


General Satisfaction 

2. I am generally satisfied with the service I receive from OEP. 
Convenience 


SD D N A SA 
1 2 3 4 5 


3. OEP personnel are easily accessible at appropriate times to satisfy my requirements. 

4. I know the right person in OEP that I should contact for my problem. 

Communication 

5. OEP keeps me informed on the status of my project. 

6. I receive clear and accurate explanation on the procedures and limitations of the OEP 
services. 

7. I am satisfied with the professionalism of contact personnel at OEP. 

8. I believe that OEP personnel understand my specific needs. 

9. I believe that OEP personnel have my best interest in heart. 


1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 


Responsiveness 

10. OEP responds to my complaints and rectifies my problems. 

1 1 . I am satisfied with the time that is required to return my phone call. 

12. I am satisfied with the time that is required to respond to my inquiries in: 

a. Industrial hygiene 

b. Environmental compliance 

c. Hazardous chemicals 

d. Health physics 

e. Chemical sampling and analysis 


1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
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Flexibility 

13. lam satisfied with the ability of OEP to cope with mistakes and unforeseen problems. 

14. lam satisfied with the management of OEP in assigning the right person for the job. 
Reliability 

15. OEP performs the service right the first time. 

16. OEP performs its services at the time it promises to do so. 

17. OEP should increase its support in the following areas: 

a. Industrial hygiene 

b. Environmental compliance 

c. Hazardous chemicals 

d. Health physics 

e. Chemical sampling and analysis 


1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 



18 . What specific thing can OEP do to increase your 
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satisfaction with this organization? 


19 . What aspect of service from OEP is not covered in this questionnaire? 


Additional Comments: 
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Dear Customer: 


You have been identified as one of the Test Installations Division (TID) customers. Your candid and honest 
opinion is needed to improve and maintain the quality of our services. Please take a moment and complete the 
enclosed survey and return it to me by Friday. April 9. 1993 at Mail Stop 501-8. If you wish to remain anonymous, 
please remove this sheet and mail the remainder. If you have any questions please call me at 3-6753. 

Thank You, 


James Afarin 
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CUSTOMER OPINION SURVEY 


Thank you for participating in this survey. To better serve you, we would like to know your opinion of the quality 
of service provided by the Test Installations Division (TID). TID provides mechanical, electrical, electronic and 
laboratory technical support for all the Center s research activities. The products and services of this division are: 
Installation of research rigs and supporting systems, modifying and repairing existing research rigs, providing aircraft 
maintenance and laboratory support. There are no right or wrong answers; however, your honest opinions are very 
important. If you wish to make additional comments, please use the space provided at the end of the survey. 
Please evaluate these statements using the following scale: 

1- 1 Strongly Disagree with this statement (SD). 4- 1 Agree with this statement (A). 

2- 1 Disagree with this statement (D). 5-1 Strongly Agree with this statement (SA). 

3- 1 Neither agree nor disagree with this statement (N). 

Please identify the branch that you deal with the most. If you deal with more than one branch, please assign a 
percentage of interaction to each branch. 


Orq. Code 

Branch Name 

Branch Chief 

Managers 

7205 

Aircraft Maintenance Branch 

Edward N. Hejnal 

Reitenbach 

7210 

Altitude, Icing & Engine 
Components Branch 

Dalgleish 

Emerson; Shivak 

7230 

Engine Research Branch 

Albergottie 

Lapka; Dorony; Pamer; Cerino 

7240 

Wind Tunnels Branch 

Houghtlen 

Gary Wolf; Giomini 

7250 

Energy & Spacecraft Branch 

Cery 

Naglowsky; White 

7260 

Communications & Electronics 
Branch 

Reddish 

Kostyack; Travis; Etzler; Antczak 

7280 

Materials Development Branch 

Shepherd 

Kelbach; Gross; Schneider 

7290 

Materials & Engine Components 
Branch 

Petraus 

Geil; Aron; Al Wolfe 
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TIP QUESTIONNAIRE 


1 . During the last six months, I received services from TID times. 

General Satisfaction 

2. I am generally satisfied with the service I receive from TID. 

Availability 

3. TID personnel are available at appropriate times to satisfy my requirements. 

4. TID has adequate tools and equipment to provide the services that I need. 
Communication 

5. I am satisfied with the professionalism of contact personnel in TID. 

6. I am satisfied with the clear communication line to the management of TID. 

7. TID keeps me informed on the status of my project. 

8. I receive clear explanation on the procedures and limitations of the TID services. 
Responsiveness 

9. I am satisfied with the time that is required to: 

a. Return my phone call. 

b. Receive technical support from TID in laboratories. 

c. Receive technical support from TID in facilities. 

d. Receive hardware installation services from TID. 

e. Receive hardware maintenance services from TID. 

f. Receive operation services from TID. 


SD [ 
1 2 

1 2 
1 2 

1 2 
1 2 
1 2 
1 2 


1 2 
1 2 
1 2 
1 2 
1 2 
1 2 


i N A SA 
3 4 5 

3 4 5 
3 4 5 

3 4 5 
3 4 5 
3 4 5 
3 4 5 

3 4 5 
3 4 5 
3 4 5 
3 4 5 
3 4 5 
3 4 5 
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10. II is easy to obtain: 

a. Technical support from TID in facilities. 

b. Technical support from TID in laboratories. 

c. Hardware installation services from TID. 

d. Hardware maintenance services from TID. 

e. Operation services from TID. 

Reliability 

11. TID performs its services at the time it promises to do so. 

12. TID performs the service correctly the first time. 

13. TID makes products that meet the pre-established design characteristics. 

14. TID makes products that meet the pre-established operating characteristics. 

Flexibility 

15. lam satisfied with the management of TID in assigning the right person for the job. 

16. I can contact the right person in TID for my problem. 

17. I am satisfied with the ability of TID to cope with mistakes and unforeseen problems. 

18. I am satisfied with TID’s efforts in customizing services for my specific needs. 
Competence 

19. I am satisfied with the technical proficiency of the TID personnel in providing: 

a. Technical support from TID in facilities. 

b. Technical support from TID in laboratories. 

c. Hardware installation services from TID. 

d. Hardware maintenance services from TID. 

e. Operation services from TID. 


1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 


Demand 


20. TID should increase its support in the following areas: 

a. Technical support from TID in facilities. 

b. Technical support from TID in laboratories. 

c. Hardware installation services from TID. 

d. Hardware maintenance services from TID. 

e. Operation services from TID. 

21 What specific thing can TID do to increase your satisfaction with this organization? 


22. What aspect of service from TID is not covered in this questionnaire? 


Additional Comments: 
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Dear Customer: 

You have been identified as one of the Facilities Operation Division (FOD) customers. Your candid and honest 
opinion is needed to improve and maintain the quality of our services. Please take a moment and complete the 
enclosed survey and return it to me by Friday. April 23, 1993 at Mail Stop 501-8. If you wish to remain anonymous, 
please remove this sheet and mail the remainder. If you have any questions please call me at 3-6753. 

Thank You, 


James Afarin 
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CUSTOMER OPINION SURVEY 


Thank you for participating in this survey. To better serve you, we would like to know your opinion of the quality 
of service-provided by the Facilities Operation Division (FOD). FOD provides technical and engineering support for 
the operating, modification, maintenance, installation and repair of research support facilities, institutional facilities 
and various systems. The products and services of FOD are: Engineering support, research support, facilities 
maintenance, ground maintenance, custodial services, trash removal, security and fire protection. There are no right 
or wrong answers; however, your honest opinions are very important. If you wish to make additional comments, 
please use the space provided at the end of the survey. Please evaluate these statements using the following scale. 

1- 1 Strongly Disagree with this statement (SD). 4- 1 Agree with this statement (A). 

2- 1 Disagree with this statement (D). 5- 1 Strongly Agree with this statement (SA). 

3- 1 Neither agree nor disagree with this statement (N). 

Please identify the branch that you deal with the most. If you deal with more than one branch, please assign a 
percentage of interaction to each branch. 


Org. Code 

Branch Name 

Branch Chief 

7301 

Management & Project Support Office 

Thomas 

7302 

Engineering Support Office 

Mainthia 

7303 

Central Control Operations Office 

Wroblewski 

7304 

Research Systems Operations Office 

Norton 

7310 

System Controls Engineering Branch 

Webb. Jr. 

7320 

Center Operations Engineering Branch 

Vega 

7330 

Research Systems Engineering Branch 

Horansky 

7340 

Electrical Power Systems Branch 

Torn 

7360 

Project Management Branch 

Craddock 

7370 

Institutional Support Branch 

Jones 

7380 

Lewis Fire Department 

Allen, Jr. 

7390 

Security Branch 

Mohr 
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FOD QUESTIONNAIRE 


1. During the last six months, I received services from FOD — 
General Satisfaction 

2. I am generally satisfied with the service I receive from FOD. 
Convenience 


SD D N A SA 
1 2 3 4 5 


3. FOD personnel are easily accessible at appropriate time to satisfy my requirements. 12 3 5 

4. | know the right person in FOD that I should contact for my problem. 1 2 3 4 5 

Communication 

• 1 2 3 4 5 

5. FOD keeps me informed on the status of my project. 

6. | receive clear and accurate explanation on the procedures and limitations of the FOD 1 2 3 4 5 


services. 

7. I am satisfied with the professionalism of contact personnel at FOD. 

8. I believe that FOD personnel understand my specific needs. 

9. I believe that FOD personnel have my best interest in heart. 
Responsiveness 

10. FOD responds to my complains and rectifies my problems. 

11. I am satisfied with the time that is required to return my phone cad. 

12. I am satisfied with the response time in the following area: 

a. Management & Project Support 

b. Engineering Support 

c. Central Control Operation 

d. Research Systems Operations 

e. System Controls Engineering 

f. Center Operations Engineering 

g. Research Systems Engineering 


1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
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h. Electrical Power Systems 

i. Project Management 

j. Institutional Support 

k. Fire Department 

l. Security 
Flexibility 

13. lam satisfied with the ability of FOD to cope with mistakes and unforeseen problems. 

14. lam satisfied with the management of FOD in assigning the right person for the job. 
Reliability 

1 5. FOD performs the service right the first time. 

16. FOD performs its services at the time it promises to do so. 

Demand 

17. FOD should increase its support in the following areas: 

a. Management & Project Support 

b. Engineering Support 

c. Central Control Operation 

d. Research Systems Operations 

e. System Controls Engineering 

f. Center Operations Engineering 

g. Research Systems Engineering 

h. Electrical Power Systems 

i. Project Management 

j. Institutional Support 

k. Fire Department 

l. Security 


1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 


1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 


1 2 3 4 5 
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18. What specific thing can FOD do to increase your satisfaction with this organization? 


19. What aspect of service from FOD is not covered in this questionnaire? 


Additional Comments: 



Dear Customer: 


You have been identified as one of the Fabrication Support Division (FSD) customers. Your candid and 
honest opinion is needed to improve and maintain the quality of our services. Please take a moment and complete 
the enclosed survey and return it to me by Friday, April 9, 1993 at Mail Stop 501-8. If you wish to remain 
anonymous, please remove this sheet and mail the remainder. If you have any questions please call me at 3-6753. 


Thank You, 


James Afarin 
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CUSTOMER OPINION SURVEY 


Thank you for participating in this survey. To better serve you, we would like to know your opinion of the 
quality of service provided by the Fabrication Support Division (FSD). FSD is charged with instrumentation and 
fabrication of research hardware. This division satisfies the needs of experimental research by utilizing both an 
in-house workforce and outside contractual efforts. The products and services of this division are: Hardware 
fabrication and consultation service. There are no right or wrong answers; however, your honest opinions are very 
important. If you wish to make additional comments, please use the space provided at the end of the survey. 
Please evaluate these statements using the following scale: 

1- 1 Strongly Disagree with this statement (SD). 4- 1 Agree with this statement (A). 

2- 1 Disagree with this statement (D). 5- 1 Strongly Agree with this statement (SA). 

3- 1 Neither agree nor disagree with this statement (N). 


Please identify the branch that you deal with the most. If you deal with more than one branch, please assign a 
percentage of interaction to each branch. 


OrqCode 

Branch Name 

Branch Chief 

7410 

Fabrication Procurement Branch 

William M. Pollman 

7420 

Inspection and Material Processing 

August R. Scarpelli 

7430 

Research Instrumentation 

Frank V. Slam 

7440 

Machining 

Gerald L Matusik 

7450 

Model Development 

Peter J. Murray 

7460 

Metal Fabrication 

Gerald A. Marquis 
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FSD QUESTIONNAIRE 


1 . During the last six months, I received services from FSD times. 

General Satisfaction 

2. I am generally satisfied with the service I receive from FSD. 

Communication 


3. I am satisfied with the professionalism of contact personnel in FSD. 

4. I am satisfied with the clear communication line to the management of FSD. 

5. FSD keeps me informed on the status of my project 

6. I receive clear and accurate explanations of the terms and limitations of FSD services. 
Responsiveness 

7. I am satisfied with the time that is required to receive: 

a. Inspection and Material Processing 

b. Fabrication Procurement 

c. Research Instrumentation 

d. Machining 

e. Model Development 

f. Metal Fabrication 

8. I can obtain services easily in: 

a. Inspection and Material Processing 

b. Fabrication Procurement 

c. Research Instrumentation 

d. Machining 

e. Model Development 


1 2 
1 2 
1 2 
1 2 


1 2 
1 2 
1 2 
1 2 
1 2 
1 2 

1 2 
1 2 
1 2 
1 2 3 
1 2 3 


i A SA 
3 4 5 

3 4 5 
3 4 5 
3 4 5 
3 4 5 

3 4 5 
3 4 5 
3 4 5 
3 4 5 
3 4 5 
3 4 5 

I 4 5 
I 4 5 
i 4 5 
4 5 
4 5 


f. Metal Fabrication 


1 2 3 4 5 
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Reliability 

9. FSD performs its services at the time it promises to do so. 

10. FSD performs the service correctly the first time. 

11. FSDmakes products that meet the pre-established design characteristics. 

12. FSD makes products that meet the pre-established operating characteristics. 

13. FSD fabricated products are durable. 

14. FSD procured products are durable. 

Perceived Quality 

15. The quality of FSD fabricated products meets my expectations. 

16. The quality of FSD procured products meets my expectations. 

Flexibility 

17. I can contact the right person for my problem easily. 

18. lam satisfied with the management of FSD in assigning the right person for the job. 


Demand 

19. FSD should increase its support in the following areas: 

a. Inspection and Material Processing 

b. Fabrication Procurement 

c. Research Instrumentation 

d. Machining 

e. Model Development 


1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 


1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 


f. Metal Fabrication 


1 2 3 4 5 
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20. What specific thing can FSD do to increase your satisfaction with this organization? 


21 . What aspect of service from FSD is not covered in this questionnaire? 


Additional Comments: 
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Customer: 

You have been identified as one of the Facilities Engineering Division (FED) customers. Your candid and 
honest opinion is needed to improve and maintain the quality of our services. Please take a moment and complete 
the enclosed survey and return it to me by Friday, April 9, 1993 at Mail Stop 501-8. If you wish to remain 
anonymous, please remove this sheet and mail the remainder. If you have any questions please call me at 3-6753, 

Thank You, 


James Afarin 
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CUSTOMER OPINION SURVEY 


Thank you for participating in this survey. To better serve you, we would like to know your opinion of the quality 
of service provided by the Facility Engineering Division (FED). FED is charged with the designing and managing 
the construction of the research and institutional facilities and process systems. The products and services of FED 
are: Facilities design contract management, environmental program management, mechanical engineering design 
services, electrical engineering design services, architectural and structural design services, and construction 
management. There are no right or wrong answers: however, your honest opinions are very important. If you wish 
to make additional comments, please use the space provided at the end of the survey. Please evaluate these 
statements using the following scale: 

1- 1 Strongly Disagree with this statement (SD). 

2- 1 Disagree with this statement (D). 

3- 1 Neither agree nor disagree with this statement (N) 

Please identify the branch that you deal with the most. If you deal with more than one branch, please assign a 
percentage of interaction to each branch. 


4- 1 Agree with this statement (A). 

5- 1 Strongly Agree with this statement (SA). 


Orq Code 

Branch Name 

Branch Chief 

7601 

Management Operations Office 

Seaver 

7610 

Mechanical Engineering Branch 

Guthrie 

7620 

Electrical Engineering Branch 

Schoeffler 

7630 

Architectural-Structure Engineering Branch 

Larson 


Construction Management Branch 


7650 


Boitel, Jr. 
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FED QUESTIONNAIRE 


1 . During the last six months, I received services from FED limes. 

General Satisfaction 

2. I am generally satisfied with the service I receive from FED. 

Convenience 

3. FED personnel are easily accessible at appropriate time to satisfy my requirements. 

4. I know the right person in FED that I should contact for my problem. 
Communication 


SD D N A SA 
1 2 3 4 5 


1 2 3 4 5 
1 2 3 4 5 


5. FED keeps me informed on the status of my project 1 c 

6. I receive clear and accurate explanation on the procedures and limitations of the FED 1 2 3 4 5 


services. 

7. I am satisfied with the consideration, respect and courtesy of contact personnel at FED. 

8. I believe that FED personnel understand my specific needs. 

9. I believe that FED personnel have my best interest in heart. 


Responsiveness 

10. FED responds to my complains and rectifies my problems. 

11. lam satisfied with the time that is required to return my phone call. 

12. I am satisfied with the time that is required to respond to my inquiries in: 

a. Facilities design contract management 

b. Environmental program management 

c. Mechanical engineering design services 

d. Electrical engineering design services 

f. Architectural and structural design services 

g. Construction management 
Flexibility 


1 2 3 4 5 
1 2 3 4 5 

1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
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13. I am satisfied with the ability of FED to cope with mistakes and unforeseen problems. 1 2 3 4 5 

14. I am satisfied with the management of FED in assigning the right person for the job. 1 2 3 4 5 

Reliability 

15. FED"performs the service right the first time. 1 2 3 4 5 

16. FED performs its services at the time it promises to do so. 1 2 3 4 5 


Demand 

17. FED should increase its support in the following areas: 

a. Facilities design contract management 

b. Environmental program management 

c. Mechanical engineering design services 

d. Electrical engineering design services 

f. Architectural and structural design services 

g. Construction management 


1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
1 2 3 4 5 
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18. What specific thing can FED do to increase your satisfaction with this organization? 


19. What aspect of service from FED is not covered in this questionnaire? 


Additional Comments: 
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